Dynamic assessment for decision support

ABSTRACT

Systems, methods and computer-readable media are provided for facilitating clinical decision support and managing patient population health by health-related entities including caregivers, health care administrators, insurance providers, and patients. Embodiments of the invention provide decision support services including providing timely contextual patient information including condition risks, risk factors and relevant clinical information that are dynamically updatable; imputing missing patient information; dynamically generating assessments for obtaining additional patient information based on context; data-mining and information discovery services including discovering new knowledge; identifying or evaluating treatments or sequences of patient care actions and behaviors, and providing recommendations based on this; intelligent, adaptive decision support services including identifying critical junctures in patient care processes, such as points in time that warrant close attention by caregivers; near-real time querying across diverse health records data sources, which may use diverse clinical nomenclatures and ontologies; improved natural language processing services; and other decision support services.

CROSS REFERENCE TO RELATED APPLICATIONS

This Application is a Continuation Application of U.S. patentapplication Ser. No. 14/148,039, titled DYNAMIC ASSESSMENT FOR DECISIONSUPPORT, filed on Jan. 6, 2014, which claims the benefit of and priorityto U.S. Provisional Application No. 61/864,992, titled CLINICAL DECISIONSUPPORT AND SERVICES, filed Aug. 12, 2013, which are each herebyexpressly incorporated by reference in their entirety.

Also, much of this disclosure is shared with the following U.S. patentapplications, filed on even date herewith, (one of which is thisapplication), each of which is hereby expressly incorporated byreference it its entirety:

U.S. Application Title Attorney Dkt. No. Filing Date DYNAMICALLYDETERMINING CRNI. 188335 14,147,978 Jan. 6, 2014 RISK OF CLINICALCONDITION DECISION SUPPORT WITH CRNI. 201763 14,148,050 Jan. 6, 2014CLINICAL NOMENCLATURES DYNAMIC QUERYING WITH CRNI. 193641 14,147,991Jan. 6, 2014 CLINICAL NOMENCLATURE DECISION SUPPORT FROM CRNI. 19364214,148,002 Jan. 6, 2014 DISPARATE CLINICAL SOURCES ENHANCED NATURALCRNI. 193643 14,148,020 Jan. 6, 2014 LANGUAGE PROCESSING DYNAMIC FLOW OFCARE FOR CRNI. 193644 14,148,028 Jan. 6, 2014 DECISION SUPPORT DYNAMICASSESSMENT FOR CRNI. 193645 14,148,039 Jan. 6, 2014 DECISION SUPPORTDETERMINING NEW CRNI. 193646 14,148,046 Jan. 6, 2014 KNOWLEDGE FORCLINICAL DECISION SUPPORT USER INTERFACE FOR CRNI. 201764 14,148,059Jan. 6, 2014 CLINICAL DECISION SUPPORT

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

In brief and at a high level, this disclosure describes, among otherthings, methods, systems, computer storage media, and graphical userinterfaces for facilitating clinical decision support and managingpatient population health by health-related entities includingcaregivers, health care administrators, insurance providers, patients,and other entities. An embodiment of the invention includes a system forproviding services for mapping clinical health information across healthrecords systems and data sources; dynamically directing care processestriggered by findings at points in time to support specialists withcontextualized decision support information for determining nextactions, using information from care plans and pathways, in some cases;discovering and incorporating, into decision support services, newontologies, and behavior generation, sensory perception, and worldmodeling to achieve adaptive goals.

An embodiment includes a system for dynamically directing the careprocess for single and multi-conditions at key points in time to providedecision support using contextually intelligent aware components. Forexample, relevant labs, findings, medications, and procedures can bepresented to a user flexed or tailored to the user, such as auser-specialty, role, venue, clinical condition(s), or other attributes.An embodiment includes one or more software agents or software routinesimplemented across a distributed cloud-computing platform forfacilitating the services. In some embodiments, the agents or routinesare autonomous or semi-autonomous, adaptive, and capable ofmachine-learning. In so doing, embodiments can provide predictive,preventative, screening and monitoring services, in addition todiagnostic and therapeutic services, for patient conditions and eventsincluding overlapping concurrent, multi-condition and multi-diagnoses.

Additional examples of decision support services provided by embodimentsof the invention are described below in the Detailed Description. Someof these services include providing timely contextual information aboutpatients including condition risks, risk factors and relevant clinicalinformation components that are dynamically updatable; imputing missingpatient information including information needed to provide diagnoses,recommendations, or decision support; dynamically generating assessmentsfor obtaining additional patient information based on context, such ascaregiver specialty; data-mining and information discovery servicesincluding discovering new knowledge, such as new clinical variablesassociated with clinical conditions or events; identifying or evaluatingtreatments or sequences of patient care actions and behaviors, andproviding recommendations based on this information; intelligent,adaptive decision support services including identifying criticaljunctures in patient care processes, such as points in time that warrantclose attention by caregivers; near-real time querying across diversehealth records data sources, which may use diverse clinicalnomenclatures and ontologies; improved natural language processingservices; and other decision support services.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are described in detail below with reference to the attacheddrawing figures, wherein:

FIGS. 1A-1E depict aspects of an exemplary operating environmentsuitable to implement embodiments of the present invention;

FIGS. 2A-2D depict block diagrams of an exemplary systems forfacilitating clinical decision support in accordance with embodiments ofthe present invention;

FIG. 3A illustratively depicts a block diagram showing an ontologyframework in accordance with embodiments of the present invention;

FIG. 3B illustratively depicts a block diagrams of an exemplary systemfor facilitating clinical decision support in accordance withembodiments of the present invention;

FIG. 3C illustratively depicts aspects of an exemplary operatingenvironment suitable to implement embodiments of the present invention;

FIG. 3D depicts a block diagram of an exemplary systems for facilitatingclinical decision support in accordance with embodiments of the presentinvention;

FIG. 3E depict aspects of an exemplary operating environment suitable toimplement embodiments of the present invention;

FIGS. 4A-4K depict flow diagrams illustrating exemplary methods ofproviding clinical decision support and services in accordance withembodiments of the present invention;

FIGS. 5A-5F illustratively depict screen displays showing examplegraphical user interfaces for facilitating clinical decision support forpatient(s) by caregiver(s) in accordance with embodiments of the presentinvention;

FIGS. 6A-6G illustratively depict example screen displays showingaspects user interfaces for performing a query in accordance withembodiments of the present invention;

FIG. 7A illustratively depicts an example finite state machine solverspecific to a patient and suitable for use to determine a condition orrecommended treatment in accordance with embodiments of the presentinvention;

FIGS. 7B-7D illustratively depict past, present, and future potentialpatient states associated with clinical decisions for a patient; and

FIG. 8A-8D depicts illustrative screen displays of an example userinterface for presenting and receiving clinical patient information inaccordance with embodiments of the present invention.

DETAILED DESCRIPTION

The subject matter of the present invention is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventor has contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

As one skilled in the art will appreciate, embodiments of our inventionmay be embodied as, among other things: a method, system, or set ofinstructions embodied on one or more computer-readable media.Accordingly, the embodiments may take the form of a hardware embodiment,a software embodiment, or an embodiment combining software and hardware.In one embodiment, the invention takes the form of a computer-programproduct that includes computer-usable instructions embodied on one ormore computer-readable media.

Computer-readable media include both volatile and nonvolatile media,removable nonremovable media, and contemplate media readable by adatabase, a switch, and various other network devices. By way ofexample, and not limitation, computer-readable media comprise mediaimplemented in any method or technology for storing information,including computer-storage media and communications media. Examples ofstored information include computer-useable instructions, datastructures, program modules, and other data representations. Computerstorage media examples include, but are not limited toinformation-delivery media, RAM, ROM, EEPROM, flash memory or othermemory technology, CD-ROM, digital versatile discs (DVD), holographicmedia or other optical disc storage, magnetic cassettes, magnetic tape,magnetic disk storage, other magnetic storage devices, and other storagedevices. These technologies can store data momentarily, temporarily, orpermanently.

Throughout this description, several terms are used to aid theunderstanding of certain concepts pertaining to the associated systemand services. These terms intended to help provide an easy methodologyof communicating the ideas expressed herein and are not necessarilymeant to limit the scope of embodiments our technology. The following isa list of these terms:

Term Clinical concept Discretized health-related information capable ofbeing encoded. A clinical concept may include clinical variables,clinical values, clinical information elements or combinations thereof.For example, a clinical variable having a clinical value may be encodedas a single code representing the clinical variable and clinical value.Clinical variable/attribute A category or type of clinical informationabout a patient such as BP, Respiratory Rate, weight, blood glucose,sex, age, condition(s), diagnoses, or other types of clinicalinformation. Clinical value Patient-specific value associated with aclinical variable, such as 132 lbs. for weight, 32 years for age, malefor sex, 120 SBP, demographic values, diagnosis (such as yes or no;acute, moderate, or other value) Clinical information element or A pieceor component of health-related information for a component patient, suchas a lab result, finding, test, study, or other element of clinicalinformation. Clinical condition A disease, diagnosis, medical issue, ormedical event for a patient, wherein an event may include an epoch orseries of epochs. Condition risk score A likelihood or probability thata patient has or will develop a clinical condition. In some instances adegree of the patient's condition (e.g., mild, severe, or acute) is alsoconsidered. Condition risk factors A set clinical variables andassociated values for a patient that are determined to be relevant to acondition, including a decision support event, and used for determininga patient's likelihood for having or developing the condition. In someembodiments and scenarios, condition risk factors can include otherconditions. Condition program A program, such as a course, algorithm,rule(s), routine, guideline, plan, or goal for determining a patient'slikelihood of having or developing a condition. Contextual ontology Aset of clinical concepts associated with a particular context, such as acondition, which may be codified and interpretable by humans andmachines.

Regarding the use of singular and plural, we do not mean to intend anysort of strict numerical implication by using the singular or plural ofa term; similar to our lack of intent to imply the singular by using “a”or “the.” Trying to capture the plural in words or in the FIGS. wouldoften lead to wordiness. For example, though we might refer to “adatabase,” clearly we fully anticipate that such reference would beequally applicable to multiple databases. By way of another example “amemory” does not imply one single memory device.

In brief and at a high level, this disclosure describes, among otherthings, methods, systems, computer storage media, and graphical userinterfaces for providing decision support services. By way of a highlevel example, systems and methods are provided for dynamicallydirecting the care processes for single and multi-conditions at keypoints in time to provide guidance using contextually intelligent awarecomponents. For example, relevant labs, findings, medications, andprocedures are presented to me flexed to my specialty, role, venue,condition, and other attributes powered by inference engines, solvers,algorithms, rules or plans, and facilitated via healthcare agentscontinually learning. In some embodiments, the systems and methodsoperate with multiple platforms, such as Cerner's PowerChart®, MPages®,Touch apps, and our Population Health, and non-Cerner solutions. In anexample embodiment software agents collaborate together to support apractice-driven workflow. Using rules-driven decision modules andthrough machine learning algorithms, the agents come up with conclusionsthat enable caregivers to identify, predict, attribute, measure, andact. Drawing data from multiple sources like HIE, EMR, Claims, and PBMdatabases, the agents can create or arrange a longitudinal person orpatient record.

For example, in one embodiment and still at a high level, a supervisedlearning algorithm simplifies the process of standardizing proprietarycodes so that all clinical attributes are coded to appropriatestandards. Person or patient records from different systems are thenmatched using a record-linking formula that uses both clinical anddemographic attributes. These efforts result in a complete and shareableperson record. Using this longitudinal record alongside a contextualdiscern ontology framework and mathematical models, the agents producethe appropriate outcomes, such as condition risk scores. The contextualontology, which is like a language for a particular risk, that bothhumans and machines can interpret, is developed using a supervisedmathematical knowledge discovery engine. It enables the module topresent information based on the caregiver's role, venue, and patientcondition. Like a human learning from past experience, solvers androutines pore over a decade of de-identified health facts data toprovide the healthcare agents with the knowledge needed to producehelpful conclusions. Accordingly, like a highly skilled assistant,embodiments of the systems and methods present the caregiver withinformation at just the right time, along with data that is relevant tothe current context. And if something is missing that would helpcomplete the picture, some embodiments of the systems and methods evenhighlights that for the caregiver.

Continuing the example, upon opening a patient chart, the chartapplication connects with embodiments of the decision support services,which may operate in the cloud, to see if there are any new risks for aclinical condition (including a decision support event) to manage for aparticular patient or sets of patients. In some embodiments, a dynamicmenu or table of contents (TOC) indicates the risk for one or moreclinical conditions, and in some embodiments, a newly identified risk ishighlighted or colored. In some embodiments, a caregiver may be notifiedof a newly identified risk by alarm, email or text message, icon, orother technique. Upon selecting a risk of one or more conditions fromthe TOC, the user-caregiver is provided supporting details, including acondition risk score and risk factors that contributed to the score andother relevant clinical information elements. In some embodiments, thisscore can be calibrated based on the caregiver's own health system'shistorical activity and can be categorized into high, medium, or lowrisk level. In some embodiments, in addition to this score and relatedfactors that contributed to the risk score, the user interface of thedecision-support application displays the clinical components that arerelevant to the risk, role, and venue, using the list of meaningfulconcepts it built for this risk. In some embodiments, these componentsand clinical factors within each component are further customizablebefore the provider role or specific need. In some embodiments, a widevariety of such components, such as labs, findings, meds, vitals, andorders may be incorporated into the user-interface decision supportapplication and provide end users with the capability of customizing orbuilding build their own risk models.

In some embodiments, the decision support patient chart includesfunctionality for flexing or altering the information that is displayed(including what information is presented and the order, ranking, orpriority that the information is presented) based on the caregiverspecialty, condition(s), venue, or other attributes. For example, when auser opens another patient's chart, the application again uses the user'role (e.g. caregiver specialty) and venue and the person's (orpatient's) information in the cloud to see if there are new risks forconditions. In some embodiments a risk score and all the factors thatcontributed to this score are presented. In some embodiments, where thedecision support services determine that additional details might helpcomplete the picture a caregiver needs to provide appropriate care, itprovides a helpful questionnaire. In some embodiments, where a questionanswer already exists in the patient's record, it is pre-populated. Insome embodiments, the question and answer pair are associated with anappropriate industry terminology standard, such as standard clinicalconcepts, which makes it possible for additional decisionsupport-related measurements or events to be triggered. Upon submittingthe answers, an embodiment of the system sends notifications toappropriate clinicians and associated roles so they can continue care.In so doing, nothing gets lost in handoff, and each caregiver ispresented a view that is tailored or flexed to their role.

An exemplary operating environment suitable for use in implementingembodiments of the invention is described below.

Turning now to FIG. 1A there is presented an example operatingenvironment 100 suitable for practicing embodiments of the invention.Example operating environment 100 includes a computerized system forcompiling and/or running an embodiment of a method for providingdecision support in accordance with embodiments of the presentinvention. With reference to FIG. 1A, one or more electronic healthrecord (EHR) systems, such as EHR system 1 160, EHR system J 162, or EHRsystem N 164 are communicatively coupled to network 175, which iscommunicatively coupled to computer system 120. In some embodiments,components of operating environment 100 that are shown as distinctcomponents may be embodied as part of or within other components ofenvironment 100. For example, the one or more EHR systems 160-164 may beimplemented in computer system 120. Similarly, a single EHR system mayperform functions for two or more of the example EHR systems shown inFIG. 1A.

In embodiments, network 175 includes the Internet, and/or one or morepublic networks, private networks, other communications networks such asa cellular network, or similar network(s) for facilitating communicationamong devices connected through the network. Network 175 may bedetermined based on factors such as the source and destination of theinformation communicated over network 175, the path between the sourceand destination, or the nature of the information. For example,intra-organization or internal communication may use a private networkor virtual private network (VPN). Moreover, in some embodiments itemsshown communicatively coupled to network 175 may be directlycommunicatively coupled to other items shown communicatively coupled tonetwork 175.

In some embodiments, operating environment 100 may include a firewall(not shown) between a first component shown communicatively coupled tonetwork 175 and network 175. In such embodiments, the firewall mayreside on a second component located between the first component andnetwork 175, such as on a server (not shown), or reside on anothercomponent within network 175, or may reside on or as part of the firstcomponent.

Embodiments of electronic health record (EHR) systems 160, 162, or 164include one or more data stores of health records, which may be storedon storage 121, and may further include one or more computers or serversthat facilitate the storing and retrieval of the health records. In someembodiments, one or more EHR systems 160, 162, and 164 may beimplemented as a cloud-based platform or may be distributed acrossmultiple physical locations. Example embodiments of EHRs 160, 162, or164 include hospital, ambulatory, clinic, health exchange, and healthplan records systems. EHR systems 160, 162, and 164 may further includerecord systems, which store real-time or near real-time patient (oruser) information, such as wearable, bedside, or in-home patientmonitors, for example. It is further contemplated that embodiments ofEHRs 160, 162, or 164 may use distinct clinical ontologies,nomenclatures, vocabularies, or encoding schemes for clinicalinformation, or clinical terms. For example, EHR 160 may be affiliatedwith a first hospital system that uses a first nomenclature, while EHR162 may be affiliated with a second hospital system that uses a secondnomenclature. Similarly, EHR system J 162, which may be a local (withrespect to the caregiver, patient, or clinician) health record, may usea first clinical nomenclature while EHR system N 164, which may be aremotely located (with respect to the caregiver, patient, or clinician)EHR may use a second nomenclature. Further, in some embodiments EHRs160, 162, and 164 are affiliated with two or more separate health careentities that use two or more distinct nomenclatures. Although FIG. 1Adepicts multiple example EHR systems, it is contemplated that someembodiments may employ only one EHR system, or alternatively, may relyon provider clinician interface 142 for storing or retrieving patientrecord information.

Example operating environment 100 further includes provider clinicianinterface 142 communicatively coupled through network 175 to the one ormore EHRs 160, 162, and 164. Although environment 100 depicts acommunicative coupling through network 175 between interface 142 and theone or more EHRs 160, 162, and 164, it is contemplated that someembodiments of interface 142 may be directly communicatively coupled tothe EHRs. Embodiments of interface 142 may take the form of a userinterface operated by a software application or set of applications on aclient computing device such as a personal computer, laptop, smartphone,mobile computer, or tablet computing device. In one embodiment, theapplication includes the PowerChart® software, manufactured by CernerCorporation. In an embodiment, the application is a Web-basedapplication or applet.

Provider clinician interface 142 facilitates accessing and receivinginformation about a specific patient or set of patients, receivinginformation such as patient information, selections, queries, commands,or actions, and displaying results, recommendations or orders, forexample. In some embodiments interface 142 also facilitates receivingorders for the patient from the clinician/user, based on the results. Insome embodiments, interface 142 comprises a graphical user interface forfacilitating clinical decision support such as illustratively depictedin FIGS. 5A-5F, and 6A-6E. Additionally, interface 142 may use used forproviding diagnostic or update services, such as evaluating or modifyingcondition programs, prediction models associated with conditionprograms, libraries, content tables used for agents discussed inconnection to FIG. 1C, for facilitating human confirmation orcomputer-derived determinations such as actions, recommendations,diagnoses, linkages, or other such services.

Embodiments of provider/clinician interface 142 may take the form of auser interface and application, which may be embodied as a softwareapplication operating on one or more mobile computing devices, tablets,smart-phones, front-end terminals in communication with one or moreservers, back-end computing systems, laptops or other computing devices.In some embodiments, provider/clinician interface 142 includes aWeb-based application or set of applications that is usable to manageuser services provided by embodiments of the invention. In an embodimentinterface 142 includes Cerner's PowerChart application. Additionaldetails of some embodiments of interface 142 are described in connectionto user interface 2140 of FIG. 1C and FIGS. 5A-5F, and 6A-6E.

Example operating environment 100 further includes computer system 120,which may take the form of a server, and which is communicativelycoupled through network 175 to EHR systems 160, 162, and 164, storage121, and provider/clinician interface 142.

Computer system 120 comprises one or more processors operable to receiveinstructions and process them accordingly, and may be embodied as asingle computing device or multiple computing devices communicativelycoupled to each other. In one embodiment, processing actions performedby system 120 are distributed among multiple locations such as one ormore local clients and one or more remote servers. In one embodiment,system 120 comprises one or more computing devices, such as a server,desktop computer, laptop, or tablet, cloud-computing device ordistributed computing architecture, a portable computing device such asa laptop, tablet, ultra-mobile P.C., or a mobile phone.

Embodiments of computer system 120 include computer software stack 125,which in some embodiments operates in the cloud, as a distributed systemon a virtualization layer within computer system 120. Some embodimentsof software stack 125 include a distributed adaptive agent operatingsystem 129, which may be implemented as a platform in the cloud, andwhich is capable of hosting a number of services such as 122, 123, 124,126, and 128 and decision support manager services. Embodiments ofservices 122, 123, 124, 126, and 128 run as a local or distributed stackin the cloud, on one or more personal computers or servers such assystem 120, and/or a computing device running interface 142. In oneembodiment, interface 142 operates in conjunction with software stack125. Embodiments of services 122, 123, 124, 126, and 128 may be embodiedas one or more software agents, programs, applications, or routines, andin some embodiments are implemented using a BDI model, such as describedbelow.

In embodiments, variables mapping service 122 and records/documents ETLservice 124 provide services that facilitate retrieving frequent itemsets, extracting database records, and cleaning the values of variablesin records. For example, variables indexing service 122 may performfunctions for synonymic discovery, indexing or mapping variables inrecords, or mapping disparate health systems' ontologies, such asdetermining that a particular medication frequency of a first recordsystem is the same as another record system. In some embodiments, theseservices may invoke or be invoked by EMPI services 123 or agent/solversservices 126, or other software services. EMPI services 123 includeservices related to patient record linkage or master patient indexingservices, such as described in connection to EMPI component 3250 of FIG.3B. Agent/solver services 126 include services that perform statisticalsoftware operations, and include statistical calculation packages suchas, in one embodiment, the R system (the R-project for StatisticalComputing, which supports R-packages or modules tailored for specificstatistical operations, and which is accessible through theComprehensive R Archive Network (CRAN) at http://cran.r-project.org).Agent/solver services 126 are associated with framework services 128,which in an embodiment includes Apache Hadoop and in an embodimentsHbase framework and/or Apache Spark, or in another embodiment similarframeworks, including so called “BigData platforms” which are operablefor providing a distributed file system, and which in some embodimentsfacilitate provide access to cloud-based services such as those providedby Cerner Healthe Intent®.

Example operating environment 100 also includes storage (or data store)121, which in some embodiments includes patient data for a candidatepatient and information for multiple patients; variables associated withpatient recommendations; recommendation knowledge base; recommendationrules; recommendations; recommendation update statistics; an operationaldata store, which stores events, frequent itemsets (such as “X oftenhappens with Y”, for example), and item sets index information;association rulebases; agent libraries, solvers and solver libraries,such as for agent/solvers services 126, and other similar informationincluding data and computer-usable instructions; patient-derived data;and health care provider information, for example. It is contemplatedthat the term data includes any information that can be stored in acomputer-storage device or system, such as user-derived data, computerusable instructions, software applications, or other information. Insome embodiments, data store 121 comprises the data stores associatedwith the one or more EHR systems, such as 161, 162, and 164 andinterface 142. Further, although depicted as a single storage datastore, data store 121 may comprise one or more data stores, or may be inthe cloud.

Turning briefly to FIG. 1B, there is shown one example embodiment ofcomputing system 900 that has software instructions for storage of dataand programs in computer-readable media. Computing system 900 isrepresentative of a system architecture that is suitable for computersystems such as computer system 120. One or more CPUs such as 901, haveinternal memory for storage and couple to the north bridge device 902,allowing CPU 901 to store instructions and data elements in systemmemory 915, or memory associated with graphics card 910, which iscoupled to display 911. Bios flash ROM 940 couples to north bridgedevice 902. South bridge device 903 connects to north Bridge device 902allowing CPU 901 to store instructions and data elements in disk storage931 such as a fixed disk or USB disk, or to make use of network 933 forremote storage. User I/O device 932 such as a communication device, amouse, a touch screen, a joystick, a touch stick, a trackball, orkeyboard, couples to CPU 901 through south bridge 903 as well. Thesystem architecture depicted in FIG. 1B is provided as one example ofany number of suitable computer architectures, such as computingarchitectures that support local, distributed, or cloud-based softwareplatforms, and are suitable for supporting computer system 120.

Returning to FIG. 1A, in some embodiments, computer system 120 is acomputing system made up of one or more computing devices. In someembodiments, computer system 120 includes an adaptive multi-agentoperating system, but it will be appreciated that computer system 120may also take the form of an adaptive single agent system or a non-agentsystem. Computer system 120 may be a distributed computing system, adata processing system, a centralized computing system, a singlecomputer such as a desktop or laptop computer or a networked computingsystem.

In some embodiments, computer system 120 is a multi-agent computersystem with one or more software agents. Turning now to FIG. 1C, in someembodiments, computer system 120, storage 121, and software stack 125are implemented in operating environment 2100 using a multi-agentcomputer system such as exemplary computing system 2130 with one or moreagents 2135, as shown in FIG. 1C and described in greater detail below.But it will be appreciated that computing system 2130 may also take theform of a single agent system or a non-agent system, as described inother embodiments herein. In some embodiments, computer system 120 isembodied as computing system 2130, and in some embodiments, stack 125operates on an embodiment of computing system 2130. Computing system2130 may be embodied as a distributed computing system, a centralizedcomputing system, a single computer such as a desktop or laptop computeror a networked computing system, described in other embodiments herein.

In some embodiments, computing system 2130 is a multi-agent computersystem with agents 2135. Multi-agent computing system 2130 may be usedto address the issues of distributed intelligence and interaction byproviding the capability to design and implement complex applicationsusing formal modeling to solve complex problems and divide and conquerthese problem spaces. Whereas object-oriented systems comprise objectscommunicating with other objects using procedural messaging,agent-oriented systems use agents 2135 based on beliefs, capabilitiesand choices that communicate via declarative messaging and useabstractions to allow for future adaptations and flexibility. An agent2135 has its own thread of control which promotes the concept ofautonomy.

Some embodiments using multi-agent system 2130 are include capabilitiesto adapt the frequency and messages used for communication between thesystem 2130 and one or more users through user interface 2140, based onchanges to the environment and provide capabilities to filter out noisydata, thereby providing more flexible and adaptable decision-makingabilities. In some embodiments, this is accomplished by using leveragingpreceptors and effectors. Preceptors or sensors, which in someembodiments may be determined by agents, detect changes in an operatingenvironment and pass this information to the agent system. Effectors,which in some embodiments may be determined agents 2135, responddirectly to changes in an operating environment and consider goals andalternatives prior to implementing a change to the environment. Examplesof sensors are further discussed in the contexts of FIGS. 7B and 7C. Forexample FIG. 7B shows epoch A at time t, which is dependent on sensors 1to N at time t−1.

Embodiments using multi-agent system 2130 further have the capability ofsupporting intelligent information retrieval and filter out noisy dataand utilize heuristics to narrow down a search space to assist insolving complex problems. The multi-agent system 2130 facilitatesdesigning individual agent behaviors and their interactions with otheragents 2135 and with users, through user interface 2140. In someembodiments, agents 2135 encoded with both declarative and proceduralknowledge and can therefore learn by means of exploration of knowledgeand imitation of other agents, for example, by leveraging aggregation ofbottom-up and top-down modeling. In some embodiments, the agent system2130 accepts an abstract workflow and converts it into an actualexecutable workflow, by for example, using contract and negotiation inmulti-agent system 2130. The executable workflow may then leverageagents to run the actual workflow.

Furthermore, embodiments using multi-agent system 2130 coordinate theactions of the agents 2135 to cooperate to achieve common objectives,and negotiate to resolve conflicts, which allows for adaptability,flexibility, and organizational relationships. The transformation ofheterogeneous knowledge and content into homogeneous knowledge andcontent is an important trait of the multi-agent system to provideinteroperability. The multi-agent system 2130 operates to achieve itsgoals while still interacting with agents, including agents outside ofthe multi-agent system 2130 (not shown) and users at a higher degree offlexibility.

As a practical example, in some embodiments, a multi-agent system 2130can be utilized to efficiently validate theoretical improvements toprocesses for detecting certain conditions, such as improvements toprocesses for determining likelihood of sepsis, described herein. Inthis example, input may be received, including patient information 2110(described below) and one or more sets, thresholds, or ranges ofvariables, from parameters 2120 (described below), such as for exampleblood pressure, blood oxygen, temperature, or other variables used inthe process for detecting sepsis described herein. Such variable sets,threshold(s), or range(s) may be received from one or more health careproviders or from an agent, and, in some embodiments, may be specifiedin one or more content tables 2124 (described below). In someembodiments, the received variable set(s), threshold(s), or ranges maydiffer based on differing opinions, strategies, or condition-detectiontheories of the health care providers, or based on differences in thepatient information 2110 available to each health care provider.

Continuing the example, in embodiments using multi-agent system 2130,for each of the variable set(s), range(s) or threshold(s), an agent 2135may be invoked for determining likelihood of a condition, includingconditions or other clinical decision support event(s), or formonitoring the patient for likelihood of the condition or event. In someembodiments the agents work in parallel, such that each agent operateswith different set, range, or threshold values, thereby resulting inmultiple evaluations for the likelihood of the condition or event beingcarried out. In some embodiments, the results of the evaluations by theagents are compared to determine which set(s), range(s), or threshold(s)performs better for determining likelihood of the condition or event.Further, in some embodiments, multi-agent system 2130 learns the set(s),range(s), threshold(s) or other parameters 2120 and patient information2110 that are more likely to result in an accurate diagnosis ordetection of the condition or event. In some embodiments, the particularset(s), range(s), threshold(s), or other parameters 2120 which yield amore accurate determination of likelihood of the condition or event areweighted, biased, or otherwise noted for future use in evaluating apatient for risk of the condition or event. Similarly, the particularprocess followed by the agent, for diagnosis or detection of thecondition or event, which led to a more accurate result, is also noted.In this way, potential improvements in the condition detection processes(described herein, below) can be determined by experimentally modifyingthe input variable set(s), range(s), threshold(s), and other parameters2120. In some embodiments, the agent responsible for implementing animproved detection process can be simply swapped with the agent handingthe existing detection process.

In some embodiments, these agent(s) and processes, including learnedprocesses, are embodied as clinical condition programs, or conditionprograms, which may be used for evaluating patient information todetermine a patient's likelihood or risk for having or developing aparticular clinical condition or decision support event. Further, insome embodiments, an agent, which may be a dedicated agent, facilitateshandling and operation of a condition program including applying it to aset of patient information, evaluating it, and updating it when needed.In some embodiments, agents may dynamically construct a clinicalcondition program for a particular patient, based on the state of thepatient. (For example, briefly referencing the patient condition statemachine shown in FIG. 7A, a patient at a particular location or state ofthe state machine, which may correspond to multiple concurrent,operlapping conditions, may have a specific condition program generatedfor determining conditions risk scores, risk factors, and clinicalinformation relevant to the condition(s).)

In some embodiments, agents 2135 continually monitor events toproactively detect problems and leverage reasoning to react anddynamically alter a plan. Practical reasoning involves managing conflictresolution where the relevant considerations are provided by the agent'sdesires about what the agent believes. This involves deliberation bydeciding what state of affairs the agent wants to achieve usingintentions and by means-end reasoning which is how to achieve thosedesires using plans. By way of background, an intention is stronger thana desire and planning achieves designated goals. Thus in one embodiment,a basic planning module consists of goals and intentions to be achieved,actions that can be performed, and a representation of the environment.These plans can thus handle priorities, uncertainty and rewards toinfluence the actual plans. An agent has its own thread of control whichpromotes the concept of autonomy. Additional information about thecapabilities and functionality of agents and distributed multi-agentoperating systems, as they relate to our invention, is provided in U.S.patent application Ser. No. 13/250,072, filed on Sep. 30, 2011, which isherein incorporated by reference in its entirety.

Continuing with FIG. 1C, system 2130 is communicatively coupled topatient information 2110 and parameters 2120 data stores, which may beembodied as storage 121, and user interface 2140. Patient information2110 and parameters 2120 data stores are illustratively depicted in FIG.1C as comprising two data stores for conceptual purposes, but it iscontemplated that patient information 2110 and parameters 2120 may bestored across multiple records systems, including different EHR systemsand in different networks or locations or may be on a single data store,as described below.

System 2130 performs processing on patient information 2110 andparameters 2120. In some embodiments, including the practical exampledescribed above, system 2130 includes one or more agents 2135, whichprocess patient information 2110 using parameters 2120 to determinegoals, plans, patient actions, orders, patient conditions andrecommended treatments, or invoke other agents, such as agent solvers,to perform these determinations. In a further example, system 2130 mayreceive patient data 2111 in the form of a natural language narrative,such as a physician's note, and invoke a data-extraction agent, fromsolvers library 2122, to extract discretized data from the note. In someembodiments, an agent or routine of system 2130 may determine clinicalconcept code(s) associated with the discretize data and store thesecodes in the patient's health record. System 2130 may then use thediscretized data, or coded concepts, and content tables 2124 toinstantiate and apply another solver agent, such as a type of healthcareagent, from solvers library 2122 to determine a patient's condition andrecommended treatments. In some embodiments, a similar process isapplied by system 2130 to other unstructured clinical data sources, suchas studies and literature, patient provided information, raw patientdata from monitors or sensors, population studies and demographicinformation or other data sources. In so doing, unstructured clinicaldata from a variety of sources can be mapped into a standard orpreferred clinical concept nomenclature. Additional details relating toagent-driven data mapping processes are provided in the embodimentsdiscussed in connection to FIG. 3A.

Upon determining a patient's condition and recommended treatments,system 2130 might invoke an expert rules engine(s) using rules 2121,which may be embodied as other agents, to determine specific actions totake or a disposition for the patient, based on the determinedconditions and recommended treatments, in an embodiment.

System 2130 is executed by or resides on one or more processors operableto receive instructions and process them accordingly, in one embodiment,and may be embodied as a single computing device, such as computingdevice 120 of FIG. 1A, or multiple computing devices communicativelycoupled to each other. In one embodiment, processing actions performedby system 2130 are distributed among multiple locations such as a localclient and one or more remote servers or in one embodiment computingdevices or EHRs associated with different health entities. In oneembodiment, system 2130 resides on a computer, such as a desktopcomputer, laptop, tablet, or mobile or smart phone computer. Otherexample embodiments of system 2130 reside on a desktop computer, a cloudcomputer or distributed computing architecture, a portable computingdevice such as a laptop, tablet, ultra-mobile P.C., a mobile phone,wearable computer, or a combination of such devices.

Example environment 2100 shows multi-agent system 2130 communicativelycoupled to user interface 2140. In an embodiment, user interface 2140comprises provider/clinician interface 142 of FIG. 1A. In someembodiments, user interface 2140 includes functionality for providing apresentation capability and user interface to facilitate communicationwith users. For example, a user may view determined results about apatient or provide additional information such as patient information,in one embodiment. User interface 2140 may be a single device or acombination of devices and may be stationary or mobile. In someembodiments, a user interface on display device takes the forms of oneor more presentation components such as a monitor, computing screen,projection device, or other hardware for displaying output. In someembodiments, a user interface 2140 takes the form of one or morepresentation components with user input components, such as a remotecomputer, a desktop computer, laptop, PDA, mobile phone, ultra-mobilePC, computerized physician's clipboard, or similar device. In someembodiments, data elements and other information may be received fromuser interface 2140. Queries may be performed by users through userinterface 2140; additional orders, tests, feedback or other informationmay be provided through the display device to user through userinterface 2140.

As described above, Environment 2100 includes data store 2110 whichincludes patient information and data store 2120 which includesparameters. In some embodiments, data stores 2110 and 2120 comprisenetworked storage or distributed storage including storage on serverslocated in the cloud and may me embodied as storage 121 of FIG. 1A.Thus, it is contemplated that for some embodiments, the informationstored in data stores 2110 or 2120 is not stored in the same physicallocation. For example, in one embodiment, one part of data store 2110includes one or more USB thumb drives or similar portable data storagemedia. In one embodiment, data stores 2110 and 2120 are distributed atmultiple locations including locations of a plurality of medicalorganizations 206, 208, and 210 or FIG. 2 . Additionally, informationstored in data stores 2110 and 2120 can be searched, queried or analyzedusing system 2130 or user interface 2140, which is further described inconnection to FIGS. 6A-6D.

As shown in environment 2100, data store 2110 comprises informationspecific to a patient, which in some instances includes incomplete,outdated, uncertain, overlapping, and conflicting information. In anembodiment, data store 2100 may represents a patient health record,which may be embodied as a longitudinal person record. Moreover, theinformation might come from a variety of sources and/or exist in avariety of formats including for example, narratives and discretizeddata. Examples of sources can include patient data from different datasources (Data source 1-N) of FIG. 3A, such as different medicalentities, traditional hospitals, walk-in clinics, urgent carefacilities, other locations that render medical services, as well asin-home patient monitors and patient-wearable sensors or monitors. Inone embodiment, patient information includes one or more of patient data2111, patient records 2112, previously determined analysis ordisposition 2113, which are associated with the patient, recommendedtreatments 2115, previously determined patient conditions 2116, and pastactions 2118 performed for the patient. In some embodiments, patientdata 2111 can include lab results, real time or near real timeinformation such as data provided by a physician, including informationbased on observation or a patient's explanation, information provided bya sensor (for example, blood pressure or heart rate), or other patientdata. In some embodiments, patient records 2112 can include electronicmedical records (“EMR”), health information exchange (“HIE”), personalhealth record (“PHR”), patient claims, and other health recordsassociated with a patient.

In some embodiments, the component pieces of patient informationincluded in patient information 2110 may be logically stored together ina complete patient record, or may be logically connected by referencepointers or addresses to one or more different patient records. In anembodiment component portions of a health record that are determined tobe probably matches to a particular patient are provisionally linked tothe patient record. In an embodiment, the provisional linkage may beconfirmed by auditing. In an embodiment, patient information 2110 isencoded into clinical concepts using a standard or preferrednomenclature, and may associated with a master patient index (MPI).

Continuing with patient information 2110, previously determined analysisand dispositions 2113 include information relating to previous analysesperformed on a patient and previous dispositions determined for thepatient, including previous analyses and dispositions determined by wayof the multi-agent system, in some embodiments. Multiple-agent system2130 may handle a complex problem, such as determining patientconditions or recommended treatments. Each of the agents 2135 maygenerate multiple analyses and/or disposition for the patient. Forexample, as described above, agents operating in parallel and usingdifferent input parameters 2120, and in some instances different patientinformation 2110, may determine a patient's likelihood of having acondition(s) or other decision support event(s). In some embodiments, adegree of statistical certainty about a determined disposition ofanalysis may be arrived at by correlating or otherwise comparing each ofthe separate analyses and/or dispositions. More specifically, ifseparate agents 2135 each determine substantially the same analysis ordisposition using different levels of patient information from a singlepatient or from a multiple patients having similar clinical informationrelevant to the determination, then there may be a higher degree ofconfidence that the analysis or disposition is accurate, given theavailable patient information. In some embodiments, if the analysis ordisposition of the separate agents ends up being a false positive fordetection of a condition, then those agents can be designated orotherwise associated as having less effective determinationcapabilities. Similarly, where agents are more effective (i.e., moreaccurate and/or more efficient, such as agents able to perform in lesstime or with less input information) at detecting a patient's condition,then those agents can be designated or otherwise associated as havingmore effective capabilities. In some embodiments, the most effectiveagent may be swapped into (or invoked for) the condition detectionprocess. For example, in determining a patient's likelihood of having aparticular condition, the most effective agent may be invoked. In someembodiments, it is conceivable that performance or effective capabilityof an agent may be dependent on the specific patient information 2110.For example, in circumstances where a set A of patient information 2110is available, agent A-prime may have the best performance, but wherepatient information 2110 is different, such as if a set B of informationis available, then agent A-prime is less effective. But another agent,such as agent B-prime, may be more effective. Therefore, an associationcan be established of which agent is more effective, based on thespecific patient information 2110 that is available for a given patient.In one embodiment, another agent handles this association and invokesthe most capable agent based on the available patient information 2110.In another embodiment, this association is encoded as a logic rule orrules engine. In this way, system 2130 learns and also adapts to be moreeffective, based on the circumstances (such as the available patientinformation 2110). In some embodiments, feedback information is providedto a user or health care provider as to which agent or which patientinformation 2110 and/or parameters 2120 provide the most accuratedetermination. In some embodiments, the feedback is provided to asupervisor agent, which manages other agents. This feedback informationfacilitates the streamlining of care for future patients and facilitatesdetermining effective treatment care plans or actions. For example, ifthe feedback information indicates that a high probability of detectionof a condition or event, can be determined based on variables V1 and V2alone, where the range of variable V1 should be RX-RY for a certainpatient type and within a specified time window, and the threshold ofvariable V2 should be T1, and that variables V3-V5 are unnecessary, thenaccordingly, only patient information regarding variables V1 and V2 areneeded. In an embodiment, variables V1 and V2 may be determined as riskfactors for the condition, and may be specified in a condition programassociated with the condition. Thus, for a given future patient, ahealth care provider only needs to specify tests or otherwise provideinformation for variables V1 and V2. The health care provider might beable to forego expensive testing for determining variables V3-V5.

In some embodiments, a health care provider might suggest a set ofranges or thresholds for variables (as parameters 2120) or a range orthreshold that is different from what is typically used. After agentsusing the different ranges of thresholds complete analyses, such asdetermining likelihood of a particular condition, system 2130 canprovide information to the health care provider, via user interface2140, for example, as to whether the suggested ranges or thresholds weremore effective or less effective at diagnosing the patient's condition.

Recommended treatments 2115 include currently and previously recommendedtreatments for a patient. In one embodiment, this information includestime-related or near realtime data associated with the time that therecommended treatment was determined, as well as an indication ofwhether the recommended treatment has been acted upon. In oneembodiment, recommended treatments 2115 also specify how the recommendedtreatment was determined, including for example, available patientinformation, the type of solver that was applied, and the resultingpatient conditions, thereby enabling a health care provider to query therecommended treatments to see how a particular treatment was determinedor to generate a report.

Past actions 2118 include previous actions determined by the multi-agentsystem 2130. Similarly to what is described above in connection torecommended treatments 2115, past actions 2118 may include timeinformation associated with the time that the action was determined orexecuted, or may also specify how the action was determined or executed.

Data store 2120 comprises parameters and information associated with themulti-agent system 2130. Although depicted as working with a multi-agentsystem, in one embodiment, data store 2120 works with single-agentsystem parameters and information, or non-agent system parameters andinformation. In one embodiment, data store 2120 includes rules for arules engine 2121, and solvers library 2122. Rules for a rules engine2121 include a set of rules or library of rules. In one embodiment,rules 2121 are usable by an expert rules engine, such as an agent 2135in multi-agent system 2130. Alternatively, in a non-agent embodiment,rules 2121 include a library of rules usable by non-agent processes. Oneexample application of rules 2121 by an example rules engine or agentincludes determining actions or dispositions associated with a patientfrom a number of determined conditions or recommended treatments.Another example includes the mapping clinical concept codes acrossmultiple clinical nomenclatures and identifying contextual ontologies bydetermining sets of frequently associated clinical concept codes.

Solvers library 2122 includes one or more solvers, which can includenon-agent solvers, agent solvers (discussed below) or both. In someembodiments, solvers, which may also be referred to as “resolvers,” areapplied to determine one or more conditions or recommended treatmentsfor a patient. A finite state machine solver may be used to determinethe conditions and recommended treatments of a patient suffering from anumber of conditions including congestive heart failure. Solvers,including solver agents, may also invoke or apply other solvers oragents. Continuing this example, the finite state machine agent solvermay invoke a linear solver, such as a mixed integer linear solver, toevaluate each state in order to determine the patient's condition riskscore in a scenario where the patient has multiple conditions. In oneembodiment, the finite state machine returns the actual state for eachclinical condition of the patient, which is then passed on to the mixedinteger linear solver as parameters, to apply the mixed integer solverbased on the clinical state, and content tables 2124. The solverslibrary 2122 can be updated as new solvers are available. Anotherexample solver is the data-extraction solver, which is described infurther detail below. A data-extraction solver is a type of solver thatis applied to unprocessed patient information, such as a physician'snarrative or patient results data, in order to generate discretized datathat is usable for other solvers. Another example solver is for imputingvalues for missing or delayed clinical values for a particular patient,wherein the imputed values are based on values of patients with similarclinical conditions or similar clinical variables, such as determined bya frequent/relevant itemset mining solver. Yet another example solver isa patient clinical events trajectories mining solver, for identifyingpatterns or sequences of clinical events for patient, which may becompared to the patterns or sequences of other patients to find similarpatients. Additional details about sequence and/or trajectory miningsolvers are described in U.S. Provisional Application 61/798,123, filedMar. 15, 2013, which is herein incorporated by reference in itsentirety.

Additional solvers are provided in FIG. 1E, by way of example only andnot limitation. With reference to FIG. 1E, a number of example runtimeor near solvers are shown on the right-hand side of the table shown inFIG. 1E. (Although the term “runtime” is used, it is contemplated thatthe solvers shown may be used in realtime (or near realtime)patient-treatment scenarios or used offline or non-realtime scenarios.)Additional details regarding Tanimoto distance matching are discussed inconnection to FIGS. 7B and 7C, and further discussed in U.S. ProvisionalPatent Application, 61/886,457 filed Oct. 3, 2013, which is hereinincorporated by reference in its entirety. Additional details regardingthe chronbach alpha solver are described in U.S. application Ser. No.13/269,279, filed Oct. 7, 2011, which is herein incorporated byreference in its entirety.

Turning back to FIG. 1C, in some embodiments, agents 2135 facilitatesolving problems including the problems described above by employing oneor more solvers from solvers library 2122 and/or using rules 2121.Furthermore, where existing rule systems may utilize forward chaining,backward chaining and combination, agents 2135 can integrate these rulecapabilities as well as other traditional and heuristic techniques.These agents 2135, which may be referred to as agent solvers, can alsoleverage the best techniques for the problem at hand. In someembodiments, these agent solvers comprise health care agents. In someembodiments the agent solvers implement a clinical condition program.

In some embodiments, the agent solvers can register their abilities orresponsibilities to the overall system and coordinate and communicatewith other agents, users, or the overall system, to solve problems basedon their current capabilities. Still further and as described above inthe condition-detection examples, new or improved solvers, which may beintroduced at future times, are able to be leveraged by swapping outcurrent agents with new agents dynamically and without the need torecompile or reconfigure the system. Thus embodiments using multi-agentsystem 2130 can provide advantages, in some scenarios, over single-agentsystems and non-agent systems. By analogy, a single celled organism isanalogous to a single-agent system, while a complex multi-celledorganism is analogous to the multi-agent system. Accordingly, the“reasoning” capabilities of multi-agent system 2130 are superior to the“reasoning” exhibited by a single-agent system, and the multi-agentsystem 2130 is not constrained at design time and has the ability togrow and adapt over time based on future needs not anticipated at thetime of instantiation or design.

In some embodiments, agents 2135 provide enhanced decision support byusing multi-agent properties like collaboration, persistence, mobilityand distributed operation, autonomy, adaptability, knowledge andintelligence, reactive and proactive capability, reuse, scalability,reliability, maintainability, security, fault-tolerance, trust, andother primary properties. In addition, numerous secondary properties ofmulti-agents in embodiments of our invention may facilitate decisionsupport, including: reasoning, planning and learning capabilities;decentralization; conflict resolution; distributed problem solving;divide-and-conquer strategies for handling complex problems; locationtransparency; allowing for competing objects to be represented;goal-driven or data-driven including agent-to-agent or user-to-agent;time-driven; support for multiple layers of abstraction above servicesthereby providing flexibility, adaptability, and reuse andsimplification; negotiation; hierarchies having dynamicself-organization; abilities to spawn and destroy agents as needed;utilization of transient and persistent data; abilities to addressuncertain, missing or inconsistent data; sensitivity to resource andtime constraints; ontology-driven functionality; flexible run timeinvocation and planning; obligations; ability to act to achieveobjectives on behalf of individuals and organizations; organizationalinfluence; and other secondary properties. Examples of agents, which maybe used by the multi-agent systems of embodiments of our technologies,include: Interface agents; planning agents; information agents; adapterwrapper agents; filter agents; discovery agents; task agents; blackboardagents; learning agents, including supervised learning, unsupervisedlearning, reinforcement learning, for example; observer agents;inference agents; communication agents; directory agents; administratorand security agents; facilitator agents; mediator agents; and agentsolvers. Agent solvers can include, for example: Markov decisionprocessing; approximate linear programming; natural language extractionsolvers (e.g., nCode, NLP, or MLP solvers); fuzzy-neural networks,logistic and linear regression; forward-chaining inference (e.g.,data-driven); backward-chaining inference (e.g., goal-driven); inductiveinference; genetic algorithm; neural network including with geneticalgorithm for training; stochastic; self-organizing Kohenen map;Q-learning; quasi-Newton; gradient; decision trees; lower/higher boundsearch; constrain satisfaction; Naives Bayes fuzzy; LP-solver includingmixed integer multi-variable min/max solvers; Finite State Machine andHFSM; temporal difference reasoning; data mining for classification,clustering, learning and prediction; K-means; support vector machines;K-nearest neighbor classification; Tanimoto distance; C5.0; a priori;EM, simulated annealing, Tabu search, multi-criteria decision making,evolutionary algorithm, and other such solvers known by one skilled inthe art.

In some embodiments, where particular types of agent solvers are moreefficient at handling certain patient scenarios, a planning agent mayinvoke the particular type of agent solver most appropriate for thescenario. For example, a finite state machine agent solver and a linearsolver agent solver may be invoked by a planning agent, in a scenarioinvolving a patient experiencing congestive heart failure. Similarly, aplanning agent might invoke one or more agent solvers for determininglikelihood of sepsis, based on patient information 2110 available and/orthe effective capability of the agent solver(s). In some embodiments,agent solvers invoke other agent solvers as necessary.

Continuing with FIG. 1C, some embodiments of multi-agent system 2130employ decision making for applications including, for example,searching, logical inference, pattern matching and decomposition. Asubset of solvers library 2122 includes decision-processing solvers2123. Decision processing solvers 2123 are a special set of solvers usedfor decision making, although it is contemplated that in someembodiments any solvers of solvers library 2122 or solver agent maybeused for decision processing. Examples of agent decision processionapplications include: searching, including heuristic and traditionalsearching; list; constraint satisfaction; heuristic informed; hillclimbing; decision tree; simulated annealing; graph search; A* search;genetic algorithm; evolutionary algorithm; Tabu search; logicalinference; fuzzy logic; forward and backward-chaining rules;multi-criteria decision making; procedural; inductive inference; patternrecognition; neural fuzzy network; speech recognition; natural languageprocessing; decomposition; divide-and-conquer; goal tree and subgoaltree; state machine; function decomposition; pattern decomposition; andother decision-processing applications. In some embodiments, agentsdesigned or instantiated with a particular decision-processingapplication may be swapped out, in a more seamless and transparentmanner than with non-agent systems, with another agent having moreadvanced decision processing functionality as this becomes available oris needed.

Solver content-parameters 2124, which is also referred to as “contenttables” 2124, include parameters used for instantiating and applying thesolvers. Content tables 2124 provide parameters that specify informationregarding conditions, drugs, contra-indications, treatments, orders orother actions, and other parameters used in conjunction with patientinformation to determine conditions and recommended treatments. In someembodiments, content tables 2124 include information for instantiatingcondition program agents including instantiating agents to construct aspecific condition program for a given patient.

In one embodiment, content parameters 2124 are formatted as independenttables, which might take the form of a matrix, which can be maintained,updated, or swapped out with other tables, by health care providers,physicians, or experts independent of patients. For example, a contenttable may specify parameters relating to diabetes including what factorsin patient information indicate that the patient is in hypoglycemia,what factors in patient information indicate that the patient is inhyperglycemia, contra-indications, treatments such as drugs and drugdosages that should be administered, or additional testing that shouldbe ordered. In another example, content tables specify the set(s),range(s), and/or threshold(s) of variables for detecting likelihood of acondition, such as sepsis. In some embodiments, rows of a content tablecorrespond to different sets, ranges, or thresholds of variables suchthat a first agent can perform its analysis using content specified in afirst row A, and a second agent working in parallel (or the first agentat a later time) can perform its analysis using content from a row B.Further, in some embodiments, the results of analyses can be enteredinto the rows or associated with the rows. Thus, where multiple agentsare running analyses in parallel, with each agent using a different setof parameters specified in one row, the results of the row thatcorrespond to the most effective analysis may be provided to the healthcare provider or otherwise published to the outside world as the resultof the determination for whether the patient has the condition, eventhough in fact there may be multiple separate results from the differentanalyses, in some embodiments. This is because in many instances, thehealth care provider only desires to know whether the patient has aparticular condition, and doesn't care about a bunch of differentagent-generated results coming from diverse parameters 2120, some ofwhich are more accurate and some of which are better than others.

In some embodiments, content tables 2124 and patient information 2110provide the information necessary for a solver to determine patientconditions and recommended treatments. Content tables may be updatedindependently, as new information, treatments, or drugs becomeavailable.

Goals 2126 include objectives which guide the system, such asembodiments of a multi-agent, single-agent, or non-agent system 2130, inthe selection of a plan and, ultimately, the determination of whatactions to take place as a result of incoming patient data. Therefore insome embodiments, goals are based on incoming patient information. Forexample, a goal may specify “determine if patient has diabetes,” “manageconditions for sepsis,” “manage conditions for CHF while keeping otherpatient conditions stable,” or “minimize the cost of patient treatment.”In some embodiments, goals are used to motivate agents 2135.Specifically, agents 2135 operate under guidance of a goal that isconsistent with patient information when deciding what actions to take,plans to select and execute, or which solvers to invoke. Thus, any planselected and executed will be consistent with the determined goals 2126,which are based on patient information 2110. Moreover, as patientinformation 2110 changes, such as when newer or additional patientinformation 2110 becomes available or a patient's condition changesduring the course of treatment, goals 2126 may be changed or replaced.In some embodiments such as multi-agent systems operating under thebelief-desire-intention (“BDI”) model, a goal is analogous to a desire.Accordingly, in one embodiment, agents 2135 may act to fulfill a desirethat is based from a set of agent beliefs or facts determined fromavailable patient information 2110. In some embodiments, goals 2126 canbe organized in one or more sets, groups, tables, databases, orlibraries with, in some embodiments, subgroups related to similargoal-objectives; for example, a subgroup of goals may relate to handlingpatient treatment costs or treating cancer.

Plans 2128 include, in some embodiments, specific executable algorithms,instructions, schedules, or the similar plans for carrying out aspecific objective that is consistent with a determined goal 2126. Forexample in some embodiments, a clinical condition program may beimplemented as a plan. In some embodiments, plans 2128 may specifyintention or an intended outcome to be achieved that is consistent witha determined goal 2126. Plans 1228 can include sets or libraries ofplans, which in some embodiments are associated with certain goals 2126.For example, for the goal of “manage conditions for sepsis while keepingother patient conditions stable,” plans associated with this goal mayspecify actions for determining a patient's condition by examiningpatient information including blood pressure and blood oxygen. The planmay further specify recommended treatments, orders, or other plans to beexecuted. In some embodiments, plans 2128 also include planning agents,which can assist in the selection and execution of a plan. For example,a planning agent may select a plan, which in some embodiments may alsobe an agent, for treating sepsis based on patient information thatindicates sepsis; the plan may specify using solvers such as logisticalregression on the patient information to determine the patient'scondition and recommended treatment, in one embodiment.

In another example, a specific condition program plan under thecondition-detection goal, may specify using a data-extraction agent forextracting discrete data items from a physician's note written innatural language, and then instantiating one or more solver agents,which carry out the processes for determining likelihood of thecondition or of risk factors that may be present, which are used todetermine the likelihood of the condition. In one embodiment rather thanspecifying a specific solver or set of solvers to use (e.g.,data-extraction and finite state machine solvers), a plan may specify anintention, (e.g., determine patient's condition when patient informationindicates sepsis), and invoke an agent 2135 to select the best solverapplicable based on the available patient information 2110. Under theBDI model, a plan is analogous to an intention; thus a plan is sort oflike an intention to process the goal with which the plan is associated.The plan 2128 is executed to meet the goal 2126, or partially meet thegoal. In one embodiment, a planning engine is used for determining plans2128 consistent with goals 2126. The planning engine, which could be anagent, non-agent rules engine, a decision tree, or other decisionprocess, selects a plan.

Turning to FIG. 1D, an illustrative example is provided of a frameworksuitable for implementing a multi-agent system, such as computing system2130 of FIG. 1C, and is referenced generally by the number 2150. In someembodiments, framework 2150 is suitable for supporting stack 125 of FIG.1A. In some embodiments, stack 125 is built on framework 2150, and insome embodiments stack 125's services operate as software services onframework 2150. Furthermore, in some embodiments using multi-agentcomputing system 2130, these services are carried out using one or moreagents.

As shown in FIG. 1D, example framework 2150 has a layered architecture.At the lowest level depicted in the example embodiment shown in FIG. 1D,framework 2150 includes the Distributed Adaptive Agent KnowledgeOperating System (“DAAKOS”). In an embodiment, DAAKOS includes adecision-support framework. In an embodiment, DAAKOS is a multi-agentframework with heuristic, adaptive, self-optimizing and learningcapabilities and the ability to decompose complex problems intomanageable tasks to assist clinical decision making at point of care.For example, caregivers and other users can leverage this intelligentagent system to detect a change in personal health or to leverage up todate knowledge about medical conditions, preventive care, and otherrelevant interests. Accordingly, in one embodiment, DAAKOS can bethought of as an intelligent, self-learning agent system using a cloudmetaphor.

Specifically, DAAKOS utilizes multi-agents 2135 that collaborate witheach other and interface with external systems, services and users andhas the ability to monitor changes and incorporate past knowledge intodecision making in order to generate and evaluate alternative plans oradapt plans to handle conflict resolution and critical constraints. Amulti-agent virtual operating system provides efficient means to buildcomplex systems composed of autonomous agents with the ability to bereactive, persistent, autonomous, adaptable, mobile, goal-oriented,context aware, cooperative and able to communicate with other agents andnon-agents. In some embodiments, intelligence is achieved within agentsby way of support provided by a rich ontology within a semantic network.For example, a multi-level of collaborating agents 2135 allows low-levelagents to transform data so that it can be passed on to another agent,and to continue the data transformation until the data has beencompletely transformed from bits of data which may sometimes representincomplete, outdated, or uncertain data, to form a usable collection ofdata with rich meaning. In this example, when it becomes necessary toattack complex problems, the agent 2135 is permitted to constrain andnarrow its focus at an individual level to support decomposition. Domainspecific agents can be leveraged in some embodiments to use an ontologyto manage local domain-specific resources.

The DAAKOS operating system layer handles process management,scheduling, memory, resource management, Input/Output (“I/O”), security,planning, as well as other processes traditionally handled by operatingsystems, and in some embodiments includes memory, which may includeshort, intermediate, and/or long-term memory, I/O, internal agentblackboard, context switching, kernel, swapper, device resourcemanagement, system services, pager, process managers, and logging,auditing, and debugging processes. In some embodiments, the DAAKOSoperating system layer is a distributed virtual operating system.

In an embodiment, The DAAKOS layer is built upon a multi-agent frameworklayer such as JADE. In other embodiments, frameworks such as Cougaar,Zeus, FIPA-OS, or an open-agent architecture, or other suitablearchitecture may be used. In an embodiment, such as the exampleembodiment depicted in FIG. 1D, DAAKOS includes a multi-agent framework.

In some embodiments, the multi-agent framework includes the followingproperties: FIPA compliance; support for autonomous and proactive agentsand loosely coupled agents; peer-to-peer communication; fullydistributed architecture; efficient transportation of asynchronousmessages; support for white and yellow page directory services; agentlife-cycle management; agent mobility; subscription mechanism foragents; graphical tools for debugging and maintenance; support forontology and content languages; library for interaction protocol;extensible kernel for extensions to build customized framework;in-process interface for launching and control; support for runningagents on wireless mobile devices; integration with various Web-basedtechnologies; and pure Java implementation.

JADE, which is an acronym for Java Agent DEvelopment framework is amiddleware software development framework that is used for facilitatingimplementation of multi-agent systems. Specifically, the JADE platformincludes functionality which facilitates the coordination of multipleagents, and functionality for facilitating the distribution of agentplatforms across multiple machines, including machines running differentoperating systems. Moreover, JADE further includes functionality forchanging system configuration at run time by moving agents from onemachine to another, as required.

On top of the DAAKOS operating system layer, in the embodimentillustratively provided in FIG. 1D, is the DAAKOS Semantic Engine, whichprovides the platform for DAAKOS agents 2135. DAAKOS agents 2135 includecomplex agents and primitive agents. On top of the agents' layers areDAAKOS Applications. These include, for example, DAAKOS agent solverssuch as a finite state machine instantiated and executed to determine apatient's conditions and recommended treatments, transactions knowledgeengines, and other applications leveraging DAAKOS agents 2135.

Turning now to FIG. 2A, a block diagram of an exemplary system forfacilitating clinical decision support is provided and referred togenerally as system 2000. Decision support system 2000 supports multipleplatforms such as PowerChart 2060, MPages (not shown), Touch 2070,Population Health 2080, and other EMRs 2090, for example. In someembodiments, system 2000 comprises many agents, including healthcareagents 2035 and other agents (not shown) collaborating together toenable a practice-driven workflow. For example, in some embodiments,using inference engine(s), solver agent(s), algorithm(s), rule(s), orplan(s) 2010 (“solver agent(s) 2010”) which include traditionalrules-driven decision components and machine learning algorithms oragents, the healthcare agents 2035 determine outcomes 2050 to facilitatedecision support such as enabling a caregiver to identify, predict,attribute, measure, or act. The outcomes may be presented to careproviders at appropriate points in time along with data that is relevantto the current context, as described further below. Drawing data 2040from multiple sources, such as HIE, EMR, Claims, and PBM databases, theagents arrange a longitudinal person record, as further described inconnection to FIGS. 3A and 3B.

With reference to FIG. 2B, a block diagram of another aspect of anexemplary system for facilitating clinical decision support is providedand referred to generally as system 2200. The system of FIG. 2B showscloud services including web services and DAAKOS agents (health careagents), which interact with ontology, questionnaire, and conceptmapping data stores. The interaction with the questionnaire (or dynamicassessment) is further described in connection to FIG. 5E. Agentinteraction with ontology and concept mapping is further described inconnection to FIGS. 3A, 2D, and 3B. Web services also accessesquestionnaire data store for generating patient assessments, as shown inFIG. 5E. On the client side, EMR applications (such as third partyapplications, patient health chart applications, including PowerChart®and Cerner's MPages™ platform, and proxy services access the cloudservices.

With reference to FIG. 2C, a block diagram of an aspect of an exemplarysystem for processing and mapping patient data into consumable contentis provided and referred to generally as system 2300. In this example, anatural language processing engine received patient information (such asfrom one or more patient EHRs or a data stream, which may be provided bya patient sensor or provided each time a patient is assessed by acaregiver. Natural language processing engine may use one or morenatural language processing agents to process the received patientinformation into consumable content, which may be used by decisionservices discussed in connection to FIGS. 4A through 4K. In someembodiments, the consumable content is de_identified. In someembodiments, the patient information is encoded into one or moreclinical concepts, which may be translated or “mapped” to a standard oruniversal nomenclature, as described in connection to FIG. 3A, therebyrendering the content consumable by the other decision support services,applications, features, and agents described herein.

With reference to FIG. 2D, a block diagram of another aspect of anexemplary system for facilitating clinical decision support is providedand referred to generally as system 2400. System 2400 illustrates oneexample of embodiments for a decision support application and userinterface discussed in connection to FIGS. 5A-6D. In particular, in someembodiments, clinical interface 2442 comprises user interface 5142,which by be embodied as provider/clinician interface 142 of FIG. 1A.System 2450 also shows assembly component 2450, which assemblesinformation used by interface 2442. In some embodiments, assemblycomponent 2450 operates as a web server and may assemble informationinto a markup language format for delivery to a client (not shown) inthe client domain 2410 that presents interface 2442, such as via a webbrowser. In some embodiments, assembly component 2450 facilitates theflexing and contextualization of information presented in interface2442, and the presentation of assessment(s), as described further inconnection to FIG. 5A-5F, by interpreting the outcomes received fromhealth care agents/services 2035 and generating correspondinginstructions for interface 2442 to display the appropriate information.In some embodiments, assembly component also facilitates translationbetween nomenclatures, such as by invoking a concept mapping agent ormapping/translation routine when information is received from the clientdomain that is in a proprietary clinical nomenclature, before providingthe information to agents/services 2035, other cloud services (notshown); or when information being communicated to the client is in aclinical nomenclature that is not used by the client. In someembodiments, assembly component 2450 also facilitates the dynamicfeatures of interface 2442 (or user interface 5142), such asfacilitating dynamic updating of at-risk conditions, risk scores, riskfactors, or other presented information, as it changes or becomesavailable. In embodiments, assembly component 2450 may comprise morethan one component and may be located on the client domain, in the cloudor both.

Turning now to FIG. 3A, a block diagram of an exemplary concept mappingservice (or module) 3100 is shown as part of an ontology framework fordetermining contextual ontologies and consumable content such as mappedpopulation health information; mapped clinical nomenclatures orontologies; derived knowledge such as newly identified clinicalinformation including newly identified risk factors for a condition(s)or event, newly identified condition associations/patterns or eventassociations/patterns, effective/ineffective treatments, clinicalontologies; information regarding contextually relevant clinical datafor a condition or decision support event; or other health-relatedinformation derived from data sources as described herein. In someembodiments, service 3100 is embodied as one or more agents.

By way of background and not limitation, discovery of new knowledge canbe determined from data, which may come from various data sources, suchas data source 1 through data source N and including diverse datasources of unstructured data. At a high level, embodiments of mappingservice 3100 service facilitate discovery of new knowledge first byperforming concept mapping on the data to determine structured data,which may be codified with a standard clinical nomenclature. In someembodiments, as described further in connection to FIG. 3B, patientrecords from different data sources or systems are determined to have aprobability of being records for the same patient, based on clinical anddemographic attributes,. Based on this probability, patient records arematched using a record linkage algorithm or agent thereby generating aninner-operable longitudinal patient record, which may be identified orde-identified depending on the usage context.

Mapping service 3100 receives data from one or more data sources 3110and determines mapped content 3120. Examples of data sources 3110include EHRs such as EHRs 160, 162, and 164 of FIG. 1 , literature,clinical studies or trials, historical data, government records, arrestrecords, financial, income, and spending information, demographicinformation, patient provided data, patient records including labs ortest results, claims, HIE, EMR, PMB databases, other clinical records,clinical trials, information from visits to a caregiver or treatmentfacility, patient behavior-pattern information, caregivers behaviorpattern information, questionnaires or forms, or other source ofclinical data. This list is not intending to be limiting and includesany source of information which may be used for purpose of providingclinical decision support. In one embodiment a data source may providedata in real-time or near real-time. The data may be structured orunstructured. In an embodiment, unstructured data is processed usingnatural language processing or other data extraction and formattingprocesses to create structured information. In an embodiment, this isfacilitated by one or more agents. In one embodiment, service 3100stores the structured data as mapped content 3120. In one embodiment,the mapped content is mapped to a common clinical nomenclature and maybe aggregated with mapped data from other data sources.

In one embodiment, concept recognition services, which may be embodiedas one or more software services or agents operating on multi-agentsystem 2130, are utilized for discovering and mapping concept synonymiesand other processes to produce mapped content 3120. Additional detailsabout the capabilities and functionality of concept recognition as theyrelate to embodiments of our invention is provided in U.S. patentapplication Ser. No. 13/569,781, filed on Aug. 8, 2012, which is hereinincorporated by reference in its entirety.

In one embodiment, concept mapping service 3100 enables aggregation ofdata by a user using clinical terminologies that are well defined andwell understood by the health care domain. By way of example, in oneembodiment, proprietary codes specific to an EMR or EHR are mapped to anappropriate industry-understood clinical terminology. Unstructuredcontent in the form of text, clinical documents, recordings, sensordata, or other formats is discretized or parsed using a medical languageprocess and codified.

In some embodiments, the aggregated mapped data is de-identified for usein research purposes such as population health databases. In someembodiments, using the aggregated, mapped patient data, which includesinner-operable longitudinal patient records, one or more conditions orconcepts are targeted to find related concepts, such as other conditionsfrequently present when the targeted condition or concept is present inthe patient data, as shown in blocks 3130. In some embodiments,additional attributes such as patient demographic information, treatmentinformation such as the caregiver or health entity treating the patientfor example, may also be combined with the targeted condition(s) orconcept(s) to increase the specificity of the related concepts. Forexample, the targeted condition and demographic information couldinclude African American males having diabetes under age 60. As anotherexample, patients at least 3 months pregnant with income levels belowpoverty who are treated at a particular hospital and who are enrolled ina particular medication trial. In some embodiments, the targetedcondition is multiple conditions, one or more decision support events,sequence(s) or pattern of events. For example, patients who showeddiminished symptoms of a particular condition whose health recordsinclude a common set of clinical concepts such as common orders orsequence of orders, common caregiver, common demographic attributes,common co-condition or similar common clinical concepts.

At block 3140, in some embodiments mapping service 3100 performs ameasure of internal consistency and association rules. In someembodiments this comprises performing statistical analyses on identifiedrelated concepts to determine measure(s) of association between arelated concept and targeted condition or concept. In some embodiments,related concepts like insulin and diabetes can be processed at block3140 using mathematical and machine learning algorithms or agents tofind the concepts that are relevant and contextual to the point of care,shown at block 3150.

In some embodiments of service 3100, outliers are managed at 3860.Outliers can include identified related concepts occur below astatistical sample size to be determined statistical association, forexample, or other outlying concepts. In some embodiments, managingoutliers includes identifying outliers and providing them as input intothe processes of service 3100 used to determine concepts or conditionsrelated to a target condition, at block 3130.

At 3170, service 3100 provides consumable content, available to many ofthe services, systems, applications, and platforms described herein,that may be used at the point of care. In some embodiments, prior toproviding content at 3170, the machine processed concepts are curatedand/or verified by clinical experts to create consumable content.

In some embodiments, mapping service 3100 may be used to determine acontextual ontology, or language of concepts, about a particularcondition, concept, including a risk, that both humans and machines caninterpret. The contextual ontology may be developed using a supervisedmathematical knowledge discovery engine, which may be embodied as anagent, routine, or set of agents or routines. The discovery engineenables service 3100 to provide consumable content that can be presentedbased on a caregiver's role, venue, patient condition, or otherattributes, as determined in connection to 3140, and as described inconnection to FIGS. 5A-5F.

In some embodiments, agent solvers 2010, from FIG. 2A, analyze years ofde-identified health facts data to provide the health care agents withthe knowledge needed to produce helpful conclusions. In this way, healthcare agents are capable of providing a caregiver with information at theright time including data that is relevant to the current condition andcontext. In some embodiments, missing information that would otherwisebe helpful, such as clinical information needed to determine a conditionrisk, is highlighted for the caregiver. In some embodiments, anassessment is dynamically generated and provided to the appropriatecaregiver to facilitate obtaining the missing information. In someembodiments, missing information for the patient is imputed usinginformation from similar patients identified by consumable content 3100.In some embodiments, the imputed information may be statisticallyimputed using information from similar patients, and statisticalprocesses such as Monte Carlo simulation. Further, in some embodiments,imputed patient information is highlighted, color-coded, presented witha notice, or otherwise presented to a caregiver so as to enable acaregiver to identify imputed patient clinical data from actual patientclinical data.

Additional details regarding aspects of embodiments of mapping service3100 are provided in U.S. patent application Ser. No. 13/645,896, filedon Oct. 5, 2012, which is herein incorporated by reference in itsentirety.

With reference to FIG. 3B, a block diagram of one example embodiment ofa system for facilitating clinical decision support is shown andreferred to generally as system 3200. Each block in example decisionsupport system 3200 is intended to show a component or step forproviding decision support services 3201, including point-of-careservices, mapping services, natural language processing services, newknowledge discovery, described in connection to FIGS. 4A-8D.

At mapping component 3240, a supervised learning algorithm facilitatesthe process of standardizing proprietary codes so that all clinicalinformation elements or concepts are coded to appropriate standards,such as described in connection to FIG. 3A. In some embodiments, mappingcomponent 3240 uses solver(s), inference engine(s), algorithm(s),rule(s), or plan(s) 3230 (“solver agent(s) 3230”) (such as described inmapping services 3100 of FIG. 3A) to process data from data sourcesincluding population health data such as Cerner Health Facts 3220,literature and published evidence 3210, or other data sources (notshown) including real-time and near real-time or streaming data sources.

At EMPI component 3250, patient records and patient information fromdifferent systems or data sources are matched using a record-linkingalgorithm, that uses both clinical and demographic attributes, in someembodiments, thereby creating a complete shareable or inner-operablepatient record. In an embodiment, the complete patient record madeshareable in that it is mapped from a proprietary or first nomenclatureto a standard clinical coding nomenclature (second nomenclature). Asdescribed above in connection to FIG. 3A and patient information datastore 2110 of FIG. 1C, the component pieces of patient recordinformation may be logically stored together in a complete patientrecord, or may be logically connected by reference pointers or addressesto one or more different patient records. In an embodiment componentportions of a health record that are determined to be probable matchesto a particular patient are provisionally linked to the patient record.In an embodiment, the provisional linkage may be confirmed by auditing.Additional information regarding of embodiments of EMPI component 3250,including identifying and linking patient records as they relate to theinvention, are provided in U.S. patent application Ser. No. 13/874,961filed on May 1, 2013, and U.S. Provisional Patent Application,61/886,457 filed Oct. 3, 2013, each of which are herein incorporated byreference in its entirety.

Using the longitudinal patient record alongside a contextual decisionsupport ontology framework and mathematical models or other servicesused by solver agents 3230, health care agents produce the appropriateoutcomes for decision support services 3201.

Turning now to FIG. 3C, an aspect of an exemplary operating environmentshowing an example of relationships between certain elements of decisionsupport services is shown and referred to generally as environment 3300.Environment 3300 includes a plurality of clinical attributes/variables(“CV”) 3310 (including demographic attributes), which may represent acategory or type of clinical information about a patient such as BP,Respiratory Rate, weight, blood glucose, sex, age, condition(s),diagnoses, patient attribute, demographic, or other type of clinicalinformation. A clinical variable 3130 may be associated with a clinicalvalue 3220 or clinical value 3225, which may be a patient-specific valuefor a clinical variable, such as 132 lbs. for weight, 32 years for age,male for sex, or 120 SBP. In some instances, clinical value 3225 variesas a function of time. For example, a clinical variable for white bloodcell may have clinical values that vary in time, which may be providedby a series of lab results. While a clinical value 3320 such as “male”or “female” for the clinical variable “sex” would not be expected tochange for a patient. Clinical variables and clinical values may beassociated with clinical concepts, and in some embodiments are capableof encoding into clinical concept codes.

Clinical variables 3130 may be associated with a clinical condition orevent 3330. For example, the condition 3130 for diabetes may beassociated with clinical variables 3130 for blood glucose and weight. Insome embodiments, these associations represent discovered associationsdetermined via mapping service 3100 of FIG. 3A. For example, newclinical variables associated with a “target” condition or event, in theform of “related concepts” are discovered, thereby forming newlyidentified relationships between a clinical condition/event 3330 and theclinical variable.

In some embodiments, condition/event program 3340 (condition program3340) includes one or more plans (such as plan 2128 discussed inconnection to FIG. 1C), algorithm(s), rules, courses, or guidelines fordetermining a patient's likelihood of having or developing one or moreconditions or decision support events such as condition/event 3330. Insome embodiments, condition/event program 3342 addresses multipleconditions or events, which may be concurrent and overlapping. In someembodiments, condition program 3340 (or 3342) may be consideredassociated with a particular state of the patient. For example, brieflyreferencing the patient condition state machine shown in FIG. 7A, apatient at a particular location or state of the state machine, whichmay correspond to multiple concurrent, overlapping conditions, may havea specific condition program for determining conditions risk scores,risk factors, and clinical information relevant to the condition(s). Insome embodiments, the condition program is operated by an agent, whichmay be an agent dedicated to that particular condition program (such asa sepsis detection agent described in the example provided in connectionto FIG. 1C.), and in some embodiments, the condition program may bedynamically created or selected from a library, based on the particularstate of the patient. For example, a specific condition program may beused for the pregnant patient with heart failure (shown in the statemachine of FIG. 7A). In this manner, condition programs may be tailoredto the particular patient, rather than the condition itself. Moreover,as described in connection to FIG. 1C, a condition program may beembodied as a learning, adaptive agent or agent-driven process. Specificcondition programs may be evaluated and updated by swapping them outwith superior performing programs.

Some embodiments of condition program 3340 (and 3342) identify a set ofrisk factors used for determining the particular condition(s) orevent(s). Risk factors comprise a set of clinical concepts or clinicalvariables (with clinical values) that are determined to be relevant to acondition, including a decision support event, that may be used fordetermining a patient's likelihood for having or developing thecondition. In some embodiments and scenarios, condition risk factors caninclude other conditions. For example, a condition program fordetermining a patient's likelihood of CHF readmission might include COPDand diabetes as risk factors. While both the diabetes and COPD riskfactors might also have dedicated condition programs, applied to thesame patient, for determining their likelihood, in some embodiments. Asnew risk factors are identified, such as by identifying concepts relatedto a targeted condition or concept, as described in connection toservice 3100 of FIG. 3A, the condition program can be updated. In someembodiments, the updated condition program is effectively “pushed out”to client sites, such as point-of-care applications by caregivers topatients. For example, suppose a newly related concept, such as aparticular clinical variable, is discovered to be statisticallyassociated with diabetes via the process described in FIG. 3A. Thisconcept related to diabetes (which may have its machine-determinedrelation audited or vetted, in some embodiments) may become a new riskfactor for diabetes. A condition program for diabetes, or a contenttable 2124 (FIG. 1C) used by an agent dynamically assembling a conditionprogram that includes diabetes, can be updated to include the new riskfactor. Where the condition program is currently in use, such as at aparticular client site, the program may be dynamically updated toinclude the new risk factor. In some embodiments, dynamic updatingoccurs when communication occurs between the client application (such asPowerChart or an application such as described in FIGS. 5A-5F) andbackend services, such as cloud services of FIGS. 2B and 2D. Theresulting update may cause a particular patient's risk score orprobability for the condition to change or may result in prompting thecaregiver to provide patient information related to the new risk factor,in some embodiments.

Turning now to FIG. 3D, with reference to FIGS. 1C, 1D, and 3A, a blockdiagram of an exemplary system 300 is shown for enabling multi-site dataapplications, such as monitoring or surveillance using data fromseparate data sources, and decision support for a patient's medicalcare. A patient information data store 302 is illustrated that storesinformation including patient records, such as electronic medicalrecords (EMR) that can be accessed by various applications. Forinstance, a particular patient's EMR may be accessed by one or moreproviders associated with different medical organizations. In someembodiments, data store 302 includes the data store of patientinformation 2110 of FIG. 1C. Further, in some embodiments, a patient'sEMR may not be accessed by a particular provider at a particular medicalorganization, but may be populated by patient information received fromthat provider at the medical organization. For example, multiple medicalorganizations, including hospitals, clinics, doctors' offices, etc., maytreat the same patient at some point in time, but may not share a commonmedical record system, and thus may not all have access to the patient'sEMR. Instead, these medical organizations may send patient informationto a location that stores the patient's EMR so that this information canbe populated into the EMR. This patient information may also bemonitored and used to determine whether the patient is at risk ofdeveloping a particular disease or condition using an appropriatecondition program.

The patient information can be returned by one of several methods. Forinstance, the updating component 304, in one embodiment, acts as acrawler and actually reaches into another medical organization's medialrecord system to pull relevant information for a particular patient whois being monitored or who may be monitored in the future for being atrisk for a particular disease or condition. In some embodiments,functions of updating component 304 are facilitated by one or moreagents 2135. The updating component 304, similarly, may query themedical organizations medical record system to obtain this patientinformation. Using this method, the crawler may include a program orapplication that tells it exactly what type of information to retrieve.Alternatively, the updating component 304 may not have the capability orpermission to crawl for patient information, but may receive patientinformation such that it is the responsibility of the medicalorganization treating the patient to send the patient information to theupdating component 304. Using either method, the updating component 304eventually receives patient information for concept mapping. A conceptrecognition component 306 performs synonymic discovery and is generallyresponsible for reconciling terms used by the various medicalorganizations, and in some embodiments is facilitated by one or moreagents 2135. For instance, if a first medical organization calls a whiteblood cell count test WBC and a second medical organization calls thesame test WC, the concept recognition component 306 would have thisinformation stored to determine that both terms are referring to thesame test. In some instances, the concept recognition component 306reconciles the test results themselves, such as if two different medicalorganizations use a different measuring system. In some embodiments,concept recognition component 306 is embodied as one or more softwareservices operating on multi-agent system 2130. In some embodiments,concept recognition component 306 uses aspects of mapping service 3100.Additional information about the capabilities and functionality of suchembodiments as they relate to our invention is provided in U.S. patentapplication Ser. No. 13/569,781, filed on Aug. 8, 2012, which is hereinincorporated by reference in its entirety.

In some embodiments, a logic data store 308 stores logic, such as acondition program, or content table information for facilitatingcreation of a condition program by an agent, that is used to determinewhen a patient is at risk for a particular disease or condition and mayalso include information specifying when it is the appropriate time toalert one or more medical organizations, the patient, the primary careprovider, etc., that the patient is at risk based on the patientinformation received from the multiple medical organizations. In someembodiments, logic data store 308 includes one or more components ofparameters 2120, such as solvers library 2122, rules 2121, and orcontent tables 2124. The concept recognition component 306 maycommunicate patient identifying information to the logic data store 308,including an identification of the patient.

Algorithm agent 310 is responsible for executing algorithms or logic,such as the logic stored in the logic data store 308. In someembodiments, algorithm agent is an agent 2135, which may be a solveragent(s) or an agent that invokes other solver agents. Algorithms orlogic may comprise algorithms and/or logic from solvers library 2122,rules 2121, and or content tables 2124, in some embodiments, and furthermay be handled by one or more dedicated agents. The algorithms or logicdetermines when, based on an array, a patient is at risk for developinga particular disease or condition. Algorithm agent 310 may additionallybe responsible for updating the logic based on results of patientmonitoring. For example, in embodiments wherein multiple agents areinvoked to determine when a patient is at risk for developing aparticular condition, agent 310 may swap out less-effective agents oragent 310 may facilitate updating the logic to indicate which particularagent should be used or which variable range(s) or threshold(s) shouldbe used, based on the analysis results of the parallel-operating agents.

In some embodiments, algorithm agent 310 may include a multi-agentsystem that has knowledge as to whether patients for which alerts aresent are actually diagnosed with the disease or condition for which theyare being monitored. If the percentage is low or otherwise unacceptableas to the patients being diagnosed, the criteria for being at risk forthat disease or condition may be altered such that alerts andnotifications are sent when a different set of criteria is met. Further,the individual medical organizations or health care entities may haveindividual criteria that they use to determine when a patient is atrisk, and thus when it would like to receive an alert from themonitoring system. Accordingly in some embodiments, condition programsmay be tailored to the specific venue (and patient information presentedto a caregiver, flexed to the venue). An algorithm agent 310 (or aplurality of agents 310) may monitor this information to determine whenit is appropriate to alert, notify, etc., one or more medicalorganizations or other parties involved in the medical care of thepatient. For instance, each provider with document-patient contactduring a period of time that the active risk assessment array has beenactive may be notified, in one embodiment.

Alerting service 312 receives input from algorithm agent 310 as to whenand whom to alert or notify. In an alternative embodiment, the alertingservice 312 is responsible for using inputs from the algorithm agent 310to determine when and whom to alert or notify. The alerting service 312may comprise one or more rules that allow the alerting service 312 howto determine when to communicate an alert, notification, includingupdated condition programs, risk scores, risk factors, sets of relevantclinical information elements, clinical determinations, recommendation,or other outputs (including health care agent outputs). In oneembodiment, each medical organization or healthcare entity that hasprovided patient information to the monitoring system receives an alert,notification, or update when the criteria are met for the patient beingat risk for a particular disease or condition, which may be determinedupon obtaining additional information about the patient, or when anupdate to a condition program results in an updated condition riskscore. Further, in some embodiments, the patient may be alerted ornotified via a text message, a telephone call, a letter, an e-mail,etc., so that the patient can initiate a follow-up appointment with theprimary care physician or another provider. Even further, the primarycare physician, while he or she may not have provided any patientinformation that was used in the array to determine that the patient isat risk for a particular disease or condition, may be alerted orotherwise notified. In some embodiments, the notification or alert isdisplayed in an electronic chart 322 corresponding to the patient, suchas an EMR so that it can be used for future reference by otherclinicians. In some embodiments, alerting service 312 facilitatesdisplaying alerts or notifications on a graphical user interface, suchas an embodiment of interface 142 of FIG. 1A.

While in some embodiments, the monitoring system establishes thecriteria for determining whether a patient is at risk for a particulardisease or condition, such as specified by a condition program, inanother embodiment, each medical organization may use different criteriafor determining whether a patient is at risk for a particular disease orcondition, which may be indicated in the same condition program or in aseparate condition program associated with that particular medicalorganization or health care entity. For instance, a first medicalorganization may use a heart rate criteria of above 95 beats per minute(bpm) for a patient being at risk for developing sepsis. A secondmedical organization may use a heart rate criteria of above 98 bpm(clinical value associated with a clinical variable) for a patient beingat risk for developing sepsis. When a patient's heart rate is at 96 bpmand other criteria are met for being at risk for developing sepsis, thefirst medical organization may receive an alert, notification, or update(such as a risk score update), but the second medical organization maynot receive an alert, notification, or update, in some embodiments. Inthese embodiments, the second medical organization may receive anotification indicating that the first medical organization received analert, notification, or update (such as a risk score update) based onits criteria for sepsis. This may prompt the second medical organizationto take a closer look at the patient's medical information to determinewhether it needs to take action. While there are many different ways ofimplementing an alerting service 312, the previous examples are providedas illustrations as to how the alerts and notifications may operate anddo not limit embodiments of the present invention. Other scenarios notspecifically mentioned here are contemplated to be within the scope ofthe present invention.

As shown, alerting service 312 can notify the medical organizations (orother healthcare entity) by communicating a notification to the medicalrecord system used by each medical organization. For instance, the firstmedical organization may utilize record system 1 (item 314), which has anative database 1 (item 316) that stores patient information includingEMRs for each of the medical organizations with which it operates. Thenotification may be communicated to record system 1 (item 314), and thenthe alert is sent to the particular medical organization or clinicianwithin that medical organization. Similarly, the notification may becommunicated to record system 2 (item 318), which has a native database2 (item 320) for storing patient information including EMRs for each ofthe medical organizations with which it operates. The alert ornotification may appear on the patient's EMR, or may be sent directly tothe clinician responsible for treating the patient. As shown in FIG. 3D,the medical record systems send patient information to the updatingcomponent 304. This patient information may be used, such as specifiedby a condition program, to determine whether a patient is at risk for aparticular disease or condition. While the patient information comesfrom individual medical organizations or health care entities, eachmedical organization may utilize a particular medical record system, andthus when the practitioner enters patient information into the patient'sEMR at the medical organization (e.g., hospital, urgent care, doctor'soffice), the patient information is sent to the medical record systemwhere it is stored. The alert sent from the alerting service 312 maytake many forms, including, for exemplary purposes only and notlimitation, an email, text message, telephone call, pager message, fax,or the like. Even further, the alert or notification may comprise arecommended care plan for the provider based on the patient informationreceived.

As shown in the embodiment of FIG. 3D, the concept recognition component306, the logic data store 308, the algorithm agent 310, and the alertingservice 312 are in the cloud. In some embodiments these components,services, and data store are implemented on a cloud-computing platform.Cloud computing generally refers to a way for on-demand network accessto a shared pool of configurable computing resources, including, forinstance, networks, servers, storage, applications, or services. Assuch, the components listed above in the cloud are accessible by way ofthe Internet or other network 175 of FIG. 1A. While these components areshown in the cloud, other components may also be in the cloud, althoughnot shown in FIG. 3D.

With reference to FIG. 3E, aspects of an example operating environmentare shown. FIG. 3E is intended to provide another example of anillustrative overview of how various aspects of the embodiment describedherein (such as in connection to FIGS. 4A through 5F) operate in ahealth care environment, such as shown in FIG. 3B. As shown in FIG. 3E,a patient is being treated by a caregiver (labeled “DR”). The patienthas a condition, the venue is the doctor's office, and the role of thecaregiver is a primary care physician, in this example. The role venueand condition (RVC) establish a context for evaluating the patient anddetermining treatment including order, recommendations, determinations,and the like (e.g., Ox, Rx, and Dx). FIG. 3E shows information frompatient health records (EHR data) and also claims data entering a“metadata smart layer” which may be facilitated by the processesdescribed in connection to FIG. 3A. In some embodiments, once theinformation is part of the smart layer, where it has been mapped to astandard nomenclature, indexed, made searchable, able to be queried, itis useable by services, applications, and/or agents, such as the healthcare agents shown. For example, an agent or service (or routine) maydetermine what other patients have been seen with symptoms like thepatient shown in FIG. 3E. Some agents are dedicated to particularconditions, such as the Sepsis, CHF and Diabetes patients shown, and mayuse information from the smart layer, research (which may be implementedas rules 2121, content parameters 2124, goals 2126, or plans 2128 ofFIG. 1C, for example), clinical evidence, and blind patient data todetermine patient risk for the condition. In an embodiment, some of theinformation in the smart layer is de_identified (i.e.patient-identifying information is removed and in some cases, clinicalvalues are altered so as to preclude or make it difficult to determinethe identity of a particular patient) and stored as blind patient data.

Turning now to FIGS. 4A-4K, flow diagrams are provided illustratingexemplary methods of providing clinical decision support and services.The methods use embodiments of systems and services described herein,such as concept mapping services, condition programs, and healthcareagents, for example. Some embodiments of these methods facilitatedecision making by dynamically displaying patient information based ontreatment session context, such as the role of the caregiver orcaregiver specialty (e.g., cardiologist, urologist, social worker,etc.), the treatment venue, such as a research hospital, walk-in clinic,emergency room, and one or more conditions or clinical decision supportevents associated with the patient. Accordingly, in some embodiments,the information presented flexes or changes based on this context. Forexample, a cardiologist treating a heart-attack victim in an ER may beshown a particular type of patient information such as patient vitals,medications, and previous heart condition diagnoses; while anendocrinologist treating a diabetic patient at a research hospital maybe shown clinical studies in which the patient is enrolled and patientcompliance with prescribed condition management regimens, for example.Further, it is contemplated that in some embodiments, informationpresented flexes based on user-caregiver preferences. For example, aparticular caregiver may always prefer to see patient vitals. Stillfurther, in some embodiments, user-caregiver interaction with anapplication, such as described in connection to FIGS. 5A-5F, is receivedand processed to determine preferences associated with particularcontexts, including role, venue, and condition. Thus, for example, afirst-time user endocrinologist may be presented with information basedon the learned preferences of other endocrinologist or further, based onthe preferences of other endocrinologist treating similar patients insimilar venues. In this manner, embodiments of the system are made to beadaptive and capable of learning. In some embodiments, this data miningof user interactions is facilitated by an agent or software routine.

With reference to FIG. 4A, a first flow diagram is provided whichdepicts an embodiment of a method for facilitating clinical decisionsupport through a user interface, such as shown in FIG. 5A, which isgenerally referred to herein as 4000. Method 4000 comprises, at step4010, communicating a portion of patient health data, the portion ofpatient health data including a set of clinical concepts for a patientencoded in a first nomenclature associated with a healthcare entity. Insome embodiments, the communicated data may be received by a backendserver or agent, and may include receiving in the first nomenclature:(1) one or more clinical conditions for the which the patient may be atrisk; (2) a computed likelihood of the patient having each of the one ormore clinical conditions; (3) information identifying a set of riskfactors associated with each of the clinical conditions relevant to thepatient's risk for each of the clinical conditions, each risk factorcorresponding to one or more clinical concepts. At a steps 4020 through4040, at-risk clinical condition(s), computed likelihood, and set ofrisk factors are determined based on converting of the portion ofpatient clinical concepts or health data from the first nomenclature toa second nomenclature (step 4020); utilizing a condition program todetermine (1), (2), and (3) in the second nomenclature (step 4030); andconverting information representing (1), (2), and (3) from the secondnomenclature to the first nomenclature (step 4040). At a step 4050, thedetermined information is communicated to the client or application atthe point of care. At step 4060, the determined information is presentedto an end user-caregiver, such as through provider/clinician interface142 or FIG. 1A, using a graphical user interface, such as shown in FIG.5A. In some embodiments, step 4060 comprises presenting a clinicalconditions menu (or table of contents (“TOC)) presenting one or moreclinical conditions for which the patient may be at risk; and presentinga condition risk area, responsive to selection of one of the one or moreclinical conditions for which the patient may be at risk, the conditionrisk area comprising: a computed likelihood of the selected condition; aset of risk factors for the selected clinical condition, and a clinicalvalue for at least a portion of the risk factors, such as shown in FIG.5A. In some embodiments, it is contemplated that method 4000 is carriedout on a single device at the point-of-care location.

In some embodiments of method 4000, clinical concepts comprise labresults, test results, patient conditions, patient health history,patient demographic information, or other discretized health-relatedinformation; clinical conditions include a disease, diagnoses, medicalissue, or medical event; the computed likelihood is a calculatedprobability that the patient has or will develop the condition based onthe clinical concepts for the patient; risk factors comprise one or moreclinical variables of health data (e.g., clinical variables andassociated clinical values) from population of patients that aredetermined to contribute to the likelihood that a given patient has orwill develop the condition;; and a clinical information element (orcomponent) comprises specific lab results, tests, health-relatedfindings, patient conditions, patient history, or otherclinical-information component of a patient's health data.

In some embodiments, steps 4020-4040 are carried out by a software agentoperating on a distributed computing platform. In some embodiments, thepresented computed likelihood of the selected condition and set of riskfactors are dynamically responsive to changes in the condition careprogram.

Some embodiments of method 4000 further comprise presenting a clinicalinformation area for displaying a set of clinical information elementsfrom the patient's health data and associated with the selectedcondition. Some embodiments of method 4000 further comprise presenting acondition assessment area for presenting a contextually-drivenassessment based on patient information relevant to diagnosing ortreating the condition, and for receiving patient information inresponse to presenting the assessment, wherein the received informationincludes one or more clinical concepts encoded in the first clinicalnomenclature. And some embodiments of method 4000 further comprisepresenting a condition assessment area for presenting acontextually-driven assessment based on patient information relevant todiagnosing or treating the condition and determining the patientinformation is absent or stale in the patient health data.

Some embodiments of method 4000 further comprise receiving patienthealth information relevant to the one or more clinical conditions forwhich the patient may be at risk; communicating a second set of clinicalconcepts corresponding to the received patient health informationencoded in the first nomenclature; receiving in the first nomenclaturean updated computed likelihood of the patient having each of the one ormore clinical conditions; and responsive to receiving the updatedcomputed likelihood, dynamically determining the computed likelihood ofthe selected condition presented in the condition risk area.

Some embodiments of method 4000 further comprise receiving an indicationof a change in the set of risk factors for a condition; and based on theindication, dynamically updating the set of factors presented in thecondition risk area.

In another embodiment, a variation of method 4000 comprisescommunicating a portion of patient health data, the portion of patienthealth data including a set of clinical concepts for a patient encodedin a first nomenclature associated with a healthcare entity. The methodfurther comprises receiving: (1) computing instructions for determininga likelihood of the patient having one or more clinical conditions; and(2) information identifying a set of risk factors associated with eachof the clinical conditions relevant to the patient's risk for each ofthe clinical conditions, each risk factor corresponding to one or moreclinical concepts, the information encoded in the first nomenclature,wherein the computing instructions and information identifying a set ofrisk factors having been determined based on: (a) converting of theportion of patient health data from the first nomenclature to a secondnomenclature; (b) utilizing a condition program to determine (1) and (2)in the second nomenclature; and (c) converting information representing(1) and (2) from the second nomenclature to the first nomenclature.Based on the received instructions, information identifying a set ofrisk factors, and the patient health data, the method further comprisesdetermining a likelihood of the patient having each of the one or moreconditions; presenting a clinical conditions menu presenting the one ormore clinical conditions for which the patient is determined to be atrisk; and presenting a condition risk area, responsive to selection ofone of the one or more clinical conditions for which the patient isdetermined to be at risk, the condition risk area comprising: (i) thedetermined likelihood of the patient having the selected condition; (ii)the set of risk factors associated with the selected clinical condition,and (iii) a clinical value for at least a portion of the risk factors.

With reference to FIG. 4B, a flow diagram is provided which depicts anembodiment of a method for facilitating clinical decision supportthrough a user interface, such as shown in FIG. 5A, which is generallyreferred to herein as 4100. Method 4100 comprises, at step 4110,receiving a first set of information about a patient, the first set ofinformation including one or more clinical concepts encoded in a firstnomenclature. At a step 4120, determining a second nomenclaturecorresponding to the first nomenclature. At a step 4230, accessing amapping database of concept mappings between the first nomenclature andthe second nomenclature. In some embodiments, a mapping databasecomprises a record or association of clinical codes of differentclinical nomenclatures for the same clinical concept(s). At a step 4140,translating at least a portion of the one or more clinical conceptsencoded in the first nomenclature into the second nomenclature therebycreating a second set of information about the patient including one ormore clinical concepts encoded in the second nomenclature. At a step4150, accessing a library of clinical condition programs, each clinicalcondition program associated with a set of clinical concept risk factorsencoded in the second nomenclature. At a step 4160, determining one ormore clinical conditions associated with the patient based on the secondset of patient information and based on at least one of the clinicalcondition programs. At a step 4170, translating into the firstnomenclature the determined one or more clinical conditions and set ofrisk factors associated with the condition program corresponding to eachof the one or more clinical conditions. In an embodiment the samemapping database is utilized for the translating operation of step 4160.At a step 4180, presenting a user interface including a menu ofcondition items, each condition item corresponding to the determined oneor more clinical conditions, wherein the conditions are presented usingthe first nomenclature. In some embodiments of step 4180, responsive toselecting an item in the menu, displaying a risk score for the patienthaving the condition corresponding to the item, and displaying a set ofrisk factors relevant to the risk score, wherein the risk factors arepresented in the first nomenclature. In some embodiments of method 4100,mapping entries of the mapping database are determined by one or moresoftware agents such as described in connection to FIGS. 1C, 2A, 3A, 3B,and 3D.

In another embodiment, a variation of method 4100 comprises receivingpatient health data from a healthcare entity, the patient health dataincluding a set of clinical concepts encoded in a first nomenclature.The method further comprises determining a second nomenclaturecorresponding to the first nomenclature; accessing a mapping database ofmappings between the first nomenclature and the second nomenclature;from the first set of clinical concepts, determining a correspondingsecond set of clinical concepts encoded in the second nomenclature; andaccessing a first clinical condition program which uses a subset of theclinical concepts in the second set of clinical concepts to determine aprobability that the patient has a first clinical condition. Based onthe first clinical condition program, the method further comprisesdetermining that a patient has a probability for the first clinicalcondition; from the mapping database, determining a set offirst-condition information including clinical concepts encoded in thefirst nomenclature and representing the first clinical condition, thedetermined probability that the patient has the first clinicalcondition, and the subset of clinical concepts used by the firstcondition program; and providing the first-condition information to thehealth care entity.

Some embodiments of method 4100 and the variation discussed abovefurther comprise presenting in a user interface, a menu including thefirst clinical condition; and responsive to selecting the first clinicalcondition, presenting: (a) a condition risk score representing thedetermined probability that the patient has for the first clinicalcondition; (b) a set of risk factors corresponding to the subset ofclinical concepts used by the first condition program; and (c) for atleast a portion of the risk factors, a clinical value for the factor,the value determined information derived from the patient.

In some embodiments the first clinical condition program is implementedby a software agent and accessing a mapping database is facilitated by asoftware agent. In some embodiments at least one of the presented menu,the presented condition risk score, or the set of risk factors aredynamically responsive to changes in the first clinical conditionprogram.

With reference to FIG. 4C, a flow diagram is provided which depicts anembodiment of a method for performing a search query of clinicalinformation and providing query services through a user interface suchas shown in FIGS. 6A-6D, in accordance with embodiments of theinvention, and which is generally referred to herein as 4200. Someembodiments of method 4200 provide dynamic queries to users across datasourced from a plurality of records systems which may use differentnomenclatures or ontologies, wherein the queries are provided in thelocal nomenclature or vocabulary (such as a proprietary nomenclature)used by a healthcare entity associated with the user, and based on thereverse code mapping of clinical concept codes such as described inconnection to FIGS. 2A, 3A-3B, and 3D. In some embodiments, a local usercan perform a search using a concept specified in his or her localvocabulary. The query is then performed on health information using astandardized concept code corresponding to the concept and query resultsare in the standardized concept code. The query results are thenconverted and displayed back to the user in the local vocabulary.

Method 4200 comprises, at step 4210, receiving one or more search termsfor a search query. At a step 4220, from a first nomenclature ofclinical concept-codes, determining a first nomenclature concept-codeassociated with each of the one or more search terms. At a step 4230,from a second nomenclature of clinical concept-codes, determining asecond nomenclature concept-code corresponding to a first nomenclatureconcept code, for each of the determined first-nomenclature conceptcodes, thereby forming a set of second nomenclature concept-codes. At astep 4240, performing a search query based on the set of secondnomenclature concept-codes. At a step 4250, receiving from the searchquery a set of search results including result concepts that are codedbased on the second nomenclature of clinical concept-codes. At a step4260, for each of the result concepts, determining a concept-code fromthe first nomenclature that corresponds to result concept code based onthe second nomenclature. And at a step 4270, providing the set of searchresults including the result concepts, wherein the result concepts arecoded based on the first nomenclature of concept codes.

In some embodiments of method 4300, a caregiver can initiate a query byselecting (such as right-clicking or holding-down on a touch surface) anitem, such as a clinical element, presented on a graphical userinterface, such as provider/clinician interface 142 of FIG. 1A. In someembodiments, a user can query on various alternative clinical decisionevents, actions, or courses of care for a patient. For example, usingembodiments of the invention to identify patients having patient recordswith a target condition(s) and a set of concepts (including attributes)similar to a target patient, a caregiver for the target patient mightperform a query to retrieve (1) the orders related to the condition thatwere issued for other patients; (2) information about which caregiverstreated those patients; (3) how the patient condition was affectedfollowing the order; and/or (4) other future actions taken or eventsoccurring with respect to the patients. Accordingly, in this manner, thecaregiver is presented with information enabling the caregiver to choosean appropriate decision for the target patient, based on the ordersissued for the similar patients and their outcomes.

In some embodiments, the query is received as a spoken command, such asa verbal request issued by a caregiver, patient, or other user. In someembodiments, concept mapping services, mapping agent(s), or synonymyagents are used for facilitating translation from a local (or first)nomenclature to a standard (or second) nomenclature.

With reference to FIG. 4D, a flow diagram is provided which depicts anembodiment of a method for data-mining including identifying and mappingnew knowledge, such as described in connection to FIGS. 2A, 3A and 3B,in accordance with embodiments of the invention, and which is generallyreferred to herein as 4300. In some embodiments, method 4300 includesmachine learning from user-caregiver interaction of point-of-care userinterfaces, such as the graphical user interfaces described inconnection to FIGS. 5A-5F. In some embodiments, interaction-activityincluding queries, selections, recommendations, preferences,use-behavior, and patient information associated with the interaction isreceived and processed. In some embodiments, the received patientinformation is de-identified. In some embodiments, the processedinformation is structured or mapped, if necessary, such as by a mappingservice 3100, and stored in a population health database, where it maybe used for providing other services described herein, such as providingrecommendations to a caregiver based on learned knowledge from datainvolving similar patients.

Method 4300 comprises, at step 4310, presenting a first clinical userinterface associated with a first patient having a condition, the firstclinical user interface including: (a) a clinical information areahaving a plurality of clinical information elements, at least a portionof the plurality of clinical information elements being populated withstored clinical information associated with the first patient; (b) aclinical recommendations area for presenting a clinical recommendationassociated with the condition; and (c) a user-input area for receivinguser commands. At a step 4320, receiving a command to initiate aclinical decision support event associated with the first patient. At astep 4330, associating the clinical decision support event with thecondition. At a step 4340, determining a change in the condition offirst patient. In some embodiments, this determination is performedbased on monitoring the patient's condition, which may be facilitated byan agent. At a step 4350, identifying a second patient having thecondition of the first patient. In some embodiments, a second patient(or a plurality of additional patients) having the condition similar tothe first patient may be identified from consumable content 3170, usinga process, such as described in connection to FIG. 3A, for identifyingpatients having patient records with the same condition(s) and set ofconcept(s), including attributes. At a step 4360, presenting a secondclinical user interface associated with the second patient. At a step4370, based on the clinical decision support event associated with thefirst patient and the determined change in the condition of the firstpatient, determining a clinical recommendation for the second patient.And at a step 4380, presenting the clinical recommendation for thesecond patient in the second user interface.

In some embodiments of method 4300, the clinical recommendation includesclinical concepts encoded using a local vocabulary or nomenclature, andin some embodiments, the received command is received in a firstnomenclature and the clinical recommendation includes clinical conceptsencoded in a second or standard nomenclature. In some embodiments,clinical concepts associated with the first patient are encoded in afirst nomenclature and clinical concepts associated with the secondpatient are encoded in a second nomenclature.

In some embodiments of method 4300, identifying a second patient havinga condition of the first patient comprises: determining a first set ofclinical concept codes associated with the condition of the firstpatient; the first set of codes being encoded in a proprietary/firstnomenclature; determining a second set of clinical concept codes thatcorrespond to the first set of codes, the second set of codes beingencoded in a standardized/second nomenclature; and searching patienthealth records to identify a second patient having clinical conceptcodes or patient information matching the second set of clinical conceptcodes.

Some embodiments of method 4300 further comprise determining acaregiver-user's response to the recommendation for the second patientand logging information indicating the user's response. In theseembodiments, the caregiver's response may become a sensor or preceptorthat determines a future epoch. In these embodiments, the patient'soutcome following the caregiver's response may be used to weight oralter the recommendation for a third (and future patients) clinicalconcept codes or patient information matching the second patient.

With reference to FIG. 4E, a flow diagram is provided which depictsanother embodiment of a method for data-mining including identifying andmapping new knowledge, such as described in connection to FIGS. 2A, 3Aand 3B, in accordance with embodiments of the invention, and which isgenerally referred to herein as 4400. Method 4400 comprises at a step4410 presenting a first clinical user interface associated with a firstpatient having a condition, the first clinical user interface including:(a) a clinical information area having a plurality of clinicalinformation elements, at least a portion of the plurality of clinicalinformation elements being populated with stored clinical informationassociated with the first patient; (b) a clinical recommendations areafor presenting a clinical recommendation associated with the condition;and (c) a user-input area for receiving user commands. At a step 4420,receiving a command to initiate a clinical decision support eventassociated with the first patient, the event being encoded in a firstnomenclature. At a step 4430, associating the clinical decision supportevent with the condition. At a step 4440, determining a change in thecondition of first patient. At a step 4450, storing a first set ofinformation for the first patient indicative of the event, theassociation with the condition, and the change in condition, theinformation encoded in a first clinical nomenclature. At a step 4460,determining a second nomenclature corresponding to the firstnomenclature. At a step 4470, translating the first set of informationencoded in the first nomenclature into a second set of informationencoded in the second nomenclature. And at a step 4480, storing thesecond set of information in a health records database.

Some embodiments of method 4400 further comprise identifying a secondpatient having the condition of the first patient; based on the secondset of information, determining a clinical recommendation for the secondpatient; and presenting the clinical recommendation for the secondpatient. In some embodiments of method 4400 the second set ofinformation is stored as de-identified information.

In another embodiment, a variation of method 4400 may be used todetermine a metric or rating associated with a particular caregiver. Forexample, a score indicating the effectiveness or efficiency of thecaregiver at treating particular types of patients, a score reflectingan average cost or resource expenditure for treatments administered bythe caregiver the caregiver, which may be normalized to othercaregivers. The variation of method 4400 comprises identifying a firstset of patients from other caregivers with similar sets of concepts;determining whether those patients got better or worse (e.g., cumulativenumber or average sum representing a change in status of a patient'scondition); identifying a second set of patients from a targetcaregiver; determining an outcome associated with those patients, suchas whether those patients got better or worse; comparing the targetcaregiver's patient outcomes to the outcomes of the reference set ofpatients, to determine a difference, and based on the difference,determining a score for the target caregiver. In some embodiments, thecaregiver metric is based on treatment costs or use of resources by thetarget caregiver versus the costs or resource expenditure of thereference set of patients

In some embodiments wherein machine learning services or agents are usedto learn new knowledge, for scenarios where only little evidence isavailable (such as a limited number of patients) a higher weighting orpriority is placed on recent information, rather than older or morestale information. Additional examples of providing priority for recentor newer learned patient information is described in U.S. ProvisionalPatent Application 61/798,123, filed on Mar. 15, 2013, which is hereinincorporated by reference in its entirety.

Turning now to FIG. 4F, a flow diagram is provided which depicts anembodiment of a method for providing natural language detection orprocessing (NLP), and which is generally referred to herein as 4500. Insome embodiments and contexts, NLP may be referred to herein as medicallanguage processing (MLP). Some embodiments of method 4500 provideenhanced NLP by correlating internal consistencies among patients, basedfrequently occurring associations of concepts, determined from themapped content (e.g., consumable content 3170) of patient data. Forexample, suppose based on rudimentary NLP processing alone, a particularphysician note appears to specify diabetes for a first patient.Embodiments of method 4500 identify other patients having sets ofconcepts in common with the first patient, and based on thatassociation, confirm the outcome of the NLP processing. For example, ifboth the first patient and the other patients have insulin orders, andthe other patients have diabetes, then it's likely that the firstpatient has diabetes.

Method 4500 comprises, at step 4510, receiving unstructuredhealth-related data associated with a first patient, the health relateddata including a discrete element. In an embodiment, an NLP service,which may be embodied as a decoder program, software routine(s) orhealth care agent, is used in step 4510 to extract one or more discreteelements from the received unstructured health-related data. In anembodiment, the NLP service uses an open-source natural languageprocessing system such as the Apache cTAKES (clinical Text Analysis andKnowledge Extraction System). In embodiment, the NLP service is modeledon the open-source UIMA (unstructured information managementarchitecture) platform from IBM, the Open NLP natural languageprocessing toolkit, or core NLP pipeline of the Open Health NaturalLanguage Process (OHNLP) Consortium. In an embodiment, an NLP service isembodied as an agent 2135 or natural language processing agent, such asdescribed in connection to FIG. 2C.

At a step 4520, identifying a preliminary match between the discreteelement and a candidate clinical concept encoded in a firstnomenclature. At a step 4530, determining a first-patient set ofclinical concepts associated with the first patient. At a step 4540receiving structured health-related data associated with a population ofpatients the structured health related data including, for each patientin the population, a set of clinical concepts matching the first-patientset of clinical concepts. At a step 4550, determining a likelihood ofpatients in the population also having the candidate clinical concept.And at a step 4560, based on the determined likelihood, determining thatthe discrete element matches the candidate clinical concept.

In another embodiment, a variation of method 4500 may be used todetermine or identify a new ontology for a condition, which may includean emerging condition. At a first step, the variation of method 4500comprises receiving unstructured patient health related data. At a nextstep, the method comprises performing natural language processing on theunstructured data to determine one or more clinical concepts from theunstructured data, wherein at least one of the concepts corresponding toa clinical condition of the patient, and the concepts are encoded in afirst nomenclature. At a next step, associating the clinical conceptswith each other, thereby forming a set of associated clinical concepts.At a next step, determining a second (or standardized) nomenclaturecorresponding to the first nomenclature. At a next step, translating theset of associated clinical concepts into the second nomenclature. At anext step, storing the set of associated clinical concepts in the secondnomenclature. At a next step, from a database of health records,determining one or more similar sets of associated clinical concepts;and designating the clinical concepts as a contextual ontology for theclinical condition.

With reference now to FIG. 4G, a flow diagram is provided which depictsan embodiment of a method for providing decision support as it relatesto directing the flow of care for a patient, such as provided inconnection to FIGS. 7A-7D, and which is generally referred to herein asmethod 4600. In some embodiments of method 4600, a dynamic flow of careis provided (in contrast to the more rigid care plans or care pathways).For example, in some embodiments of method 4600, decision points in thecourse of care are identified and recommendations provided for nextactions to take or avoid.

Method 4600 comprises at a step 4610, from a first sequence of pastclinical events relating to a patient; determining a clinical eventpattern associated with the patient. At a step 4620, from health recorddata associated with a population of patients, determining a set ofpatient health records that include the clinical event pattern. At astep 4630, for each member in the set of patient health records,determining a second sequence of clinical health events succeeding theclinical event pattern thereby forming a set of second sequences ofclinical health events. At a step 4640, based on the set of secondsequences, determining a likelihood of a future health care event forthe patient. At a step 4650, providing a recommendation of a health careaction to take based on the likelihood of the future event.

Some embodiments of method 4600 further comprise: for each member of theset of second sequences of clinical health events, determining a cost orresource expenditure associated with the sequence; determining a measureof improvement in the health condition at the end of the sequence asindicated by the health record associated with each sequence; andranking the members of the set of sequences based on the measure ofimprovement and cost or resource expenditure. Some embodiments of method4600 further comprise providing a recommendation of a care treatmentsequence for the patient based on the ranking.

Variations of embodiments of method 4600 can provide identifyingdecision points (decision epochs or critical junctures) in caretreatment, alerting caregivers that the patient is at or approaching adecision point, and providing a recommendation; anticipating likelyfuture condition states, health-care resources needed, and costsassociated with different treatment courses for a patient; anddetermining a recommended sequence of care based on learned outcomesfrom other patients having similar concepts and other patientinformation (including resources available to the patient).

With reference now to FIG. 4H, a flow diagram is provided which depictsan embodiment of a method 4700 for providing decision support byimputing missing clinical patient data, such as patient data needed fordetermining a risk, test, or recommendation, for example, for a targetpatient, based on matching patient record information about the targetpatient to a set of similar patients having information related to themissing information. In some embodiments, the imputed patient data ispresented in a manner so as to indicate to a caregiver that it isimputed and not actual patient data, as described above.

Method 4700 comprises at a step 4710, determining that patient has aprobability for event (e.g., has a condition or risk for developingsomething). At a step 4720, determining that patient is missing clinicaldata needed for confirming the determined event probability, or fordetermining an increasing/decreasing probability. At a step 4730,determining that clinical information is not currently available for oneor more of the clinical information elements relevant to the decisionsupport event. At a step 4740, identifying a set of patients havingsimilar clinical concepts associated with the condition. At a step 4750,imputing values for target patient's clinical data based on set ofpatients clinical data. And at a step 4760, using the imputed data todetermine updated risk score/event. Some embodiments of method 4700further comprise providing an indication of a difference in targetpatient's EHR between actual patient data and imputed data (e.g. colorcoding actual vs. imputed data).

With reference now to FIG. 4I, a flow diagram is provided which depictsan embodiment of a method 4800 for providing decision support byidentifying a decision point/epoch or critical juncture at future timefor a patient, based on comparing event pattern information of thepatient to event patterns of other similar patients. In someembodiments, an alert or notification is provided to a caregiverregarding an upcoming decision point.

Method 4800 comprises at a step 4810, for a patient, determining apattern of clinical events. At a step 4820, from a population ofpatients, identifying a set of patient records with similar pattern ofclinical events. At a step 4830, for each record, determining a seriesof events preceding the pattern thereby forming a “future event series”for each record. And at a step 4840, from the set of future eventseries, identifying frequently occurring events. In some embodiments,the identified events are designated as a “trigger” for starting a careprogram, or alerting a caregiver.

Some embodiments of method 4800 further comprise clustering the patientrecords by their change in condition (better, same, worse). Someembodiments of method 4800 further comprise for each cluster of records,determining a frequently occurring events among the future event series,thereby forming a frequently occurring set associated with each recordcluster. In this manner a common step is identified for the“got-better”, “same”, or “got-worse” patients.

Some embodiments of method 4800 further comprise designating frequentlyoccurring event a critical juncture, and some embodiments of method 4800further comprise alerting caregiver of critical juncture therebyfacilitating increased attention may be administered to patient.

Some embodiments of method 4800 further comprise obtaining additionalinformation, such as spawning a dynamic assessment or prompting thecaregiver for additional information to determine a next action to takeregarding the patient's treatment. Some embodiments of method 4800further comprise selecting care-pathway or future treatment program toimplement for the patient. Some embodiments of method 4800 furthercomprise based on the frequently occurring events for each cluster,determining a recommendation for a target patient.

Some embodiments of method 4800 further comprise determining a degree ofsimilarity between a target patient and the set of patient records; andbased on the degree of similarity, which may include other clinicalvariables and patient attributes, determining a probability that thetarget patient will have one or more of the events in the future eventseries. Some embodiments of method 4800 further comprise determining acost (costs include financial, risk including risks in compliance toquotas and expectations, and opportunity costs including resourcesexpended) associated with each future event series. In some embodimentsof this method two or more the records (of sets of records) are indifferent clinical nomenclatures.

With reference now to FIG. 4J, a flow diagram is provided which depictsan embodiment of a method for providing decision support by providing anassessment to a caregiver to solicit additional information, such asdescribed in connection to FIG. 5E, and which is generally referred toherein as method 4900. In some embodiments, an assessment is dynamicallygenerated and may take the form of a context-based questionnaire. Insome embodiments, branching logic is used (for example, additionalquestions are provided based on previously received answers), clinicalorder recommendations are provided in near-real-time, based on theanswers, and the risk of the patient having a particular condition isupdated in near-realtime based on the answers. In some embodiments, thequestions may be determined from the mapped content of patient healthrecords indicating concepts that frequently occur in association with aparticular condition. In some embodiments, each question-answer paircorresponds to a coded clinical concept. In some embodiments, fuzzylogic is employed to determine questions.

Method 4900 comprises at a step 4910, receiving a first set of clinicalinformation associated with a patient from a data store. At a step 4920,based on the first set of clinical information, determining apreliminary likelihood of a clinical decision support event beingassociated with the patient. At a step 4930, determining that additionalclinical information is not currently available for one or more clinicalinformation factors relevant to the clinical decision support event. Ata step 4940, generating a patient assessment for determining at least aportion of the additional clinical information, wherein the patientassessment is generated based on a treatment-session context. And t astep 4950, providing the patient assessment to a health care providertreating the patient.

In some embodiments of method 4900, the clinical decision support eventcomprises a health condition; and the treatment-session context isdetermined based on at least one of the health care provider's clinicalspecialty, a clinical treatment-session venue, and the clinical decisionsupport event.

Some embodiments of method 4900 further comprise receiving the at leasta portion of the additional clinical information in response toproviding the assessment, and updating the likelihood of a clinicaldecision support event based on the received additional clinicalinformation. Some embodiments of method 4900 further comprisedetermining clinical advice based on the likelihood of a clinicaldecision support event. And some embodiments of method 4900 furthercomprise receiving the at least a portion of the additional clinicalinformation in response to providing the assessment, and updating thedetermined clinical advice based on the received additional clinicalinformation.

In some embodiments of method 4900, the assessment is provided via agraphical user interface such as depicted in FIG. 5E, and in someembodiments, the assessment comprises a series of questions, wherein ananswer for a first question, received via the user interface, determinesa subsequent question in the series of questions. In some embodiments ofmethod 4900, the additional clinical information includes clinicalconcepts encoded in a first nomenclature. Some embodiments of method4900 further comprise determining clinical advice based on the receivedanswer to a first question and presenting the clinical advice inproximity to the first question.

In another embodiment, a variation of method 4900 comprises receiving afirst set of clinical information associated with a patient from a datastore. Based on the first set of clinical information, determining alikelihood of a clinical decision support event being associated withthe patient. At a next step, accessing an assessment associated with theclinical decision support event, the assessment including a set ofpatient related questions. Determining from the set of questions aportion of the questions to include in a questionnaire, based on atreatment-session context and the first set of clinical information. Atnext step, generating a user interface for presenting the questionnaire;presenting the user interface to a user; and receiving a set of answers,via one or more clinical information elements of the user interface, inresponse to the portion of questions in the questionnaire.

Some embodiments of this variation further comprise based on the set ofanswers, determining an additional portion of questions from the set ofquestions to include in the questionnaire, and presenting the additionalportion of questions. Some embodiments of this variation furthercomprise receiving a second set of answers to the additional portion ofquestions; determining clinical advice based on the first and secondanswers and the first set of clinical information; and presenting theclinical advice to the user. Some embodiments of this variation furthercomprise determining a clinical concept code associated with a firstquestion and a first answer received in response to the first question;and storing the concept code in the first set of clinical information.

In some embodiments, the set of questions are determined based on healthrecords for a population of patients, wherein each health recordincludes information corresponding to the clinical decision supportevent, and in some embodiments, the set of questions is furtherdetermined based on frequently occurring clinical concepts in the healthrecords. In some embodiments, the set of questions is further determinedby a health care software agent.

With reference now to FIG. 4K, a flow diagram is provided which depictsan embodiment of a method for learning new knowledge such as new relatedconcepts, risk factors, treatments, and ontologies, such as described inconnection to FIGS. 1C, 2A, 3A-3B, and 3D, and which is generallyreferred to herein as method 40100. In some embodiments, method 40100facilitates discovering and validating latent relationships in a healthcare dataset or learning new concepts that are relevant to a particularclinical event, such as a health condition(s), based on utilizing themapped or structured clinical data from patients having the sameclinical event and/or frequently occurring related concepts. In someembodiments, the clinical event may include a health condition, multipleor concurrent conditions; one condition and a particular procedure orcourse of treatment; one condition, one procedure, and one medication;or similar combinations. Thus method 40100 may be used to determine newknowledge such as complex treatment procedures or sequences, that may beapplicable to specific patients.

Method 41000 comprises at a step 40110, receiving a target set ofclinical information associated with a target population of patientsfrom a first set of records of a first health-records system, the targetset of clinical information including one or more codified clinicalconcepts. At a step 40120, receiving a reference set of clinicalinformation associated with a reference population of patients from asecond set of records of a second health-records system; the referenceset of clinical information including codified clinical concepts. At astep 40130, based on the reference set of clinical information,determining a clinical decision support event common to the records inthe reference population of patients. At a step 40140, determiningfrequent item-sets of the clinical concepts associated with thereference population. At a step 40150, associating the frequentitem-sets of clinical concepts with the clinical decision support event.At a step 40160, performing a statistical comparison between thefrequent item-sets and the clinical concepts of the target set ofclinical information to determining a statistical measure of associationbetween the frequent item-sets and the clinical concepts of the targetset. And at a step 40170, based on the statistical measure ofassociation, determining a probability that the target population ofpatients includes the clinical decision support event.

In some embodiments of method 40100, performing a statistical comparisoncomprises: performing cluster-based matching of the frequent item-setsand the target set of clinical information to determine one or moreclusters; determining at least one measure quantifying difference for atleast one cluster; and determining a statistical measure of associationbased on the quantifying difference.

In some embodiments of method 40100, the clinical decision support eventcomprises a health condition, and in some embodiments, the clinicaldecision support event comprises a combination of health conditions orclinical procedures. In some embodiments of method 40100, the clinicalconcepts are codified using a standardized clinical nomenclature.

In another embodiment, a variation of method 40100 comprises receiving areference set of clinical information associated with a referencepopulation of patients from a plurality of health-records systems; thereference set of clinical information including codified clinicalconcepts. At a next step, based on the reference set of clinicalinformation, determining a clinical decision support event common topatients in the reference population of patients. At a next step, fromthe reference set of clinical information, determining one or more setsof frequently-occurring clinical concepts. And at a next step,associating the one or more sets of frequently occurring clinicalconcepts with the clinical decision support event thereby forming one ormore event indicators for the clinical decision support event.

Some embodiments of method 40100 or the variation further comprisereceiving a set of clinical information associated with a first patient,the clinical information including codified clinical concepts;determining a number of the indicators in set of clinical information;and based on the number of indicators, determining a likelihood that thefirst patient has the clinical decision support event. Some embodimentsof method 40100 or the variation further comprise presenting thedetermined likelihood to a user, and in some embodiments, determining arecommended clinical order for the patient based on the determinedlikelihood that the first patient has the clinical decision supportevent.

In some embodiments of method 40100 and the variation, the recommendedclinical order is determined based on the reference set of clinicalinformation associated with a reference population. Some embodiments ofmethod 40100 or the variation further comprise accessing relevantclinical information from a plurality of data stores of differentnomenclatures, and some embodiments further comprise using the learnedset of risk factors for a condition program, which may includegenerating a condition program based on the set of risk factors. Andsome embodiments further comprise using the learned set of risk factorsfor diagnosing and predicting a patient's condition.

Turning now to FIGS. 5A through 5F, a series of screen displays areprovided showing example graphical user interfaces for a decisionsupport application in accordance with embodiments of the presentinvention. With reference to FIG. 5A, a first screen display 5000 isprovided showing example graphical user interface 5142. In someembodiments, interface 5142 operates on a client device or clientlocation, such as at a point-of-care. In some embodiments user interface5142 comprises provider/clinician interface 142 of FIG. 1A. In someembodiments, interface 5142 comprises an interface for Cerner PowerChartor an electronic patient chart application interface.

Interface 5142 includes a patient component or tab 5005, for presentinginformation about a patient. Interface 5142 further includes a dynamicmenu or table of contents 5010 of conditions for the patient, includingat-risk conditions and decision support events. For example, here one ofthe conditions, indicated as item 5012 in TOC 5010, includes a risk forCHF Readmission. Interface 5142 includes an area for presenting acondition risk score 5020 associated with a condition selected from TOC5010. Additionally, interface 5142 includes an area for presenting riskfactors 5030 associated with the selected condition and used by acondition program, agent, or routine, for determining the patient's riskfor having or developing the condition (e.g., the risk score). Asdescribed above in connection to FIG. 3C, in some embodiments an updatedcondition program may be “pushed out” to or dynamically updated onclients running the decision support application. Thus, the conditionitems listed in TOC 5010, the risk score 5020, and risk factors 5030 mayvary dynamically or change in near-real time, in some embodiments.

Interface 5142 further includes caregiver-role indicator 5060, whichindicates a role of the caregiver-user. In this example, the roleindicates cardiology. The role indicator 5060 may provide information toa caregiver about the flexing (or tailoring) of the particular chart (orset of patient information) being presented, since some embodiments flexthe information based on the role.

Interface 5142 further includes an area for displaying components 5040,which include clinical information element(s) such as lab results,findings, tests, studies, or other elements of clinical information. Insome embodiments, the components or clinical information elements may beexpressed as one or more clinical concept(s). As described previously,embodiments of decision support services determine which patient data topresent based on a treatment-session context such as role, venue,condition, or other attributes, such as user-caregiver preferences.Accordingly, some embodiments of components area 5040 of interface 5142present clinical information elements in a timely, contextual manner. Insome embodiments, display component 5040 are made contextually awarethrough software agents or routines, thereby allowing the relevant labs,findings, medications, and procedures are presented to the caregiverflexed to caregiver's specialty, role (e.g., cardiology), venue,condition (e.g. diabetes), and other attributes powered solver agent(s)and facilitated via healthcare agents continually learning. For example,as described previously a cardiologist treating a heart-attack victim inan ER may be shown a particular type of patient information such aspatient vitals, medications, and previous heart condition diagnoses;while an endocrinologist treating a diabetic patient at a researchhospital may be shown clinical studies in which the patient is enrolledand patient compliance with prescribed condition management regimens.

In some embodiments, the determination of which clinical informationelements or component to present is determined through the ontologyframework described in connection to FIG. 3A.

Interface 5142 also includes assessment area 5050, which is furtherdescribed in connection to FIGS. 5B and 5E. In some embodiments,assessment area presents the dynamically generated questionnairedescribed previously.

With continuing reference to FIG. 5A and reference to screen displays of5B-5F, an example user session is described. In some embodiments, uponaccessing a patient chart, the decision support application communicateswith decision support software services such as agent/solver services126 of FIG. 1A or cloud-based (or distributed) services and determineswhether there are any new risks to manage. For example, as shown in thescreen display of FIG. 5C, a highlighted (darkened) menu item in theconditions TOC on the left indicates that the patient is at risk fordiabetes. As previously described, in some embodiments there are alsoother options for getting the caregiver's attention, such as directmessaging to the caregiver's inbox, SMS, or an icon on the task bar.

Continue with the example, upon the user selecting a particular at-riskcondition, interface 5142 presents supporting details, including a riskscore 5020 and the risk factors 5030 that contributed to the score. Insome embodiments, this score can be calibrated based on theuser-caregiver's own health system's historical activity and can becategorized into high, medium, or low risk level. In addition to thisscore and related factors, some embodiments display the clinicalcomponents 5040 that are relevant to the risk, role, and venue, using alist of meaningful concepts it built for this risk, which may bespecified in a corresponding condition program, in some embodiments. Inthis example screen display of FIG. 5C, the presented components 5040include related labs and meds. In some embodiments, these components andclinical factors within each component are further customizable based onuser preferences, the provider, role, or specific need.

Continuing the example user session, a user-caregiver can access anotherpatient's chart, such as the chart for patient Scott Myron shown in theexample screen display of FIG. 5D. The screen display of FIG. 5D furtherillustrates how the decision support application flexes to presentcontextual, timely, and relevant, and other desired information to thecaregiver. Upon opening the next patient's (Scott Myron's) chart, theapplication again uses the caregiver's role and venue and the patient'sor person's information (such as patient information 2110 described inFIG. 1C) to see if there are new risks. (As described previously, thepatient information may include local patient records, patient healthrecords in the cloud, or distributed across multiple locations.) In theexample screen display of FIG. 5D, the decision support applicationinterface presents information indicating that the particular patienthas a high risk of readmission for congestive heart failure. Thecorresponding CHF Readmission risk score and risk factors thatcontributed to this score are also presented. In this exampleembodiment, the relevant clinical findings and labs to present to thecaregiver are dynamically and contextually determined.

Continuing the example, with reference to FIGS. 5D and 5E, in someembodiments, if the decision support application concludes thatadditional details might help complete the picture needed to provideappropriate care, it may provide the user-caregiver me a helpfulquestionnaire or assessment, as described previously. (In someembodiments, missing patient information might be imputed, as describedpreviously.) In some embodiments, where a question answer already existsin the patient's record, it's pre-populated.

In this example user session, a user-caregiver indicates that thepatient has some trouble sleeping (shown in box 2). In response, theapplication adds anxiety and depression as potential conditions to aproblem list. In some embodiments, the question and answer pair areassociated with an appropriate industry terminology standard, such asstandard clinical concepts, which facilitates additional decisionsupport-related actions, measurements or events to be triggered, in somecases dynamically and automatically. Upon submitting the answers (shownin box 4), an embodiment of the system sends notifications toappropriate clinicians and associated roles so they can continue care.For example, if based on information determined by the assessment orquestionnaire, the application determines that the patient might have arisk for diabetes, then a notification may be provided to anendocrinologist and the patient's primary physician. In one embodiment,the endocrinologist receives an email, instant message, or notificationwith a link that enables the endocrinologist to access the particularpatient's chart (such as presented in the example screen displays ofFIGS. 5A-5F) by clicking on the link. In some embodiments, theparticular chart presented to the endocrinologist is flexed based on thecondition, venue, and specialty or role (endocrinology) as describedpreviously. The endocrinologist and also the patient's primary physicianrepresent caregiver-users with different roles, in therole-venue-condition context, and thus may be provided with a userinterface that is flexed or tailored to their specialty, as describedpreviously. In this manner, embodiments of the invention streamlinepatient care and reduce confusion or loss of knowledge during a patienthandoff, as each caregiver is presented a view that is tailored orflexed to their role. Examples of caregiver roles are illustrativelyprovided in FIG. 5F.

With reference to FIG. 5A, in some embodiments, the particular riskfactors 5030 (which may be conditions) may be determined for a patient,based on a condition/event program, which may be selected, modified, orbuilt, as described above. In some embodiments, the risk factors specifya clinical variable and clinical value, for example age (clinicalvariable) under one years (clinical value) might be a risk factor for aparticular condition. In some embodiments, a risk factor is determinedas positive or negative for a patient from a previous diagnosesindicated by the patient health record. For example, risk factors 5030include kidney disease, which may have been diagnosed. In someembodiments, the risk factor determination is facilitated by anassessment, which may be presented in area 5050. In some embodimentsdetermining whether a patient has risk factors is handled by thecondition program, a subroutine, or handed off to a dedicated agent, andmay include input from a caregiver(s) or soliciting input from one ormore caregivers. In some embodiments, agent(s) or condition programs, ora caregiver may determine whether a patient has a given risk factorbased on the patient's EHR (such as the presence of an ICD9 code fordiabetes indicates that the patient probably has diabetes, wherediabetes is a risk factor), based on pattern(s) in the patient's healthrecord matching criteria indicating a likelihood for the risk factor orbased on the clinical values of certain clinical variables present inthe patient health record. For example, where diabetes is a risk factor,the risk factor may be identified in a particular patient, such as“gestational diabetes” for a pregnant patient having clinical variablesindicating pregnancy and unstable blood glucose levels. In someembodiments, dedicated agents including healthcare agents or solveragents may be used to determine whether the patient has a particularrisk factor.

In some embodiments, where particular patient information is missing,such as a risk factor or clinical variables used for determining a riskfactor, the values may be imputed from the values occurring in recordsof similar patients, from consumable content 3170, as describedpreviously. In some embodiments, the missing information prompts adynamic assessment such as the questionnaire presented in area 5050.

In some embodiments the dynamic assessment takes the form of aquestionnaire, but may also take the form of prompts to appropriatecaregiver or the patient to provide needed information, or scheduling ofor requests for tests, visits, orders, or other actions to facilitateobtaining the information. In some embodiments, the appropriatecaregiver receives an email or notice soliciting the desired informationor requesting a patient consultation to determine (and provide) thedesired information. In some embodiments, a patient may have sufficientinformation for determining risk, but additional information issolicited or determined so as to facilitate making finer, more targeteddecisions.

In some embodiments the assessment is contextual based on the role (or“actor type) of the caregiver. For example, in embodiments of aquestionnaire, the specific questions provided when the caregiver is acardiologist may be different than when the caregiver is anendocrinologist, a psychologist, nurse, or a social worker. Accordingly,in some embodiments the assessment also flexes based on the role, venue,and condition. In some embodiments, the assessment uses a model similarto the SCORM (shareable content object reference model) model used bythe military, which includes contingent scripts, interactive dialogs orscropt components, and may be driven off of tables for questions. Insome embodiments the questions are handled by agents, which may utilizesolvers from solver library 2122, such as finite state machines or fuzzylogic for determining assessment questions and question orders.

In some embodiments specific questions or solicited information is basedon known questionnaires or assessments used in health care, such as theUniversity of Minnesota. In some embodiments, the questions are based onmachine learning, such as learned related concepts from mapping service3100, or from learned assessments or questions provided by othercaregiver treating patients having similar condition(s) and relatedconcepts. (For example, other patients of the same demographic as atarget patient and also having condition X, as the target patient mayhave, typically have symptom Y. Does the target patient have symptom Y?)

Questions can also come from a tables or databases and use branch logic;for example, if caregiver is X and patient has Y, then ask Z. Similarly,some embodiments of the assessment ask specific additional questionsbased on the answers to previous questions. In some embodiments an agentassembles a questionnaire.

In some embodiments, as described above, question-answer pairs arecodified as concepts of a clinical nomenclature. For example, as shownin FIG. 5B, the answer to the question regarding swelling in legs orankles, the pair “some” and “swelling in ankles or legs” can becodified. In this manner, the answers can me inputted back into mappedhealth information (that is, the newly provided data is “structured” andready for consumption). Similarly, the codified question-answers aremachine-readable, interpretable, and actionable. For example, aparticular condition program, such as the sepsis detection agentdescribed previously, may be able to readily respond to the newlyprovided information because the newly provided information is machineaccessible. In some embodiments, the information acquired by theassessment is fed into the condition program, which may result inrevising a patient's determined risk score for conditions or decisionsupport event(s). The information acquired by the assessment may also befed into the ontology framework and concept mapping service described inconnection to FIG. 3A, so that newly acquired information becomes partof the mapped content 2420 of heath information used to determine setsof concepts that are related to a target condition or concept, asdescribed in connection to FIG. 3A. In some embodiments the questionsare answered via radio buttons or check boxes or similar simpleuser-interface components, which can facilitate determining the codifiedconcept for the question-answer pair. In some embodiments, theassessment application communicates with a backend service to determinethe specific codes for the question answers.

In some embodiments, answers to questions result in specificrecommendations, as shown in FIGS. 5B and 5E (boxes 2-4). Additionaldetails describing recommendations provided in response to assessmentsand other embodiments of the present invention that determine arecommendation (including a care plan, further testing or diagnostics,treatment, dosage, or consultation, or other order or recommendation),such as a recommendation provided in response to a determined condition,determined possibility of a future condition, risk for a condition, aredescribed in U.S. patent application Ser. No. 13/646,356, filed on Oct.5, 2012, which is herein incorporated by reference in its entirety.

Turning now to FIGS. 6A-6G, a series of screen displays are providedshowing aspects of query operations in accordance with embodiments ofthe present invention. Some embodiments of decision support applicationprovide query services such as described in connection to FIG. 4C. Insome embodiments, the ontology framework described in connection to FIG.3A facilitates querying across mapped health care information orconsumable content 3170. For example, with reference to FIG. 6A-6E, auser-caregiver is interested in finding how many of his or her patientshave systolic blood pressure greater than 125 that are also diabeticwithin the scope of a potential CHF Readmission risk. The user-caregivercan access a dynamic query routine (FIG. 6A), (which may be embodied asa subroutine or separate application, and may be handled by an agent),as shown in FIG. 6A. Using the ontology created for CHF readmission, alocal, or proprietary nomenclature, the user can specify the parameters(FIGS. 6B, 6C, and 6D) and receive the results (FIG. 6E) in nearreal-time. In some embodiments, the search parameters are provided inthe local nomenclature and are first mapped to a standard nomenclatureused by the pool of information being searched, such as the consumablecontent or health facts information. In these embodiments the searchresults are then mapped or translated back to the local (or proprietary)nomenclature and the results presented to the user in that local (orproprietary) nomenclature.

With reference to FIGS. 6F and 6G, an example query operation isillustratively provided. In this example, a natural language query isperformed from a graphical user interface such as described inconnection to FIG. 5A. In this example, a query is received (FIG. 6G)for “all diabetic patients with SBP>=120 and diagnosed with COPD.” In anembodiment, the query is received via the graphical user interface by acaregiver. In an embodiment, the query is received by typing a queryinto the user interface, speaking the query, which may be converted totext, or selecting one or more clinical variables of the patient forwhich to query on. In an embodiment, a query is suggested to thecaregiver based on the available clinical information for a patient suchas clinical information elements or determined clinical conditions. Inan embodiment, the received query is parsed into concept codes (such asshown in the lower window of FIG. 6G) wherein ICD9 250 corresponds to aconcept code for diabetes, and the “event code: 7328757” representssystolic blood pressure. Accordingly, in one example embodiments forperforming a query receive the query in an unstructured format, such asspoken natural language, then using a natural language processingservice (such as described in connection to FIG. 4F) the queryinformation is parsed and converted to clinical concept codes. Adetermination may be performed to identify the nomenclature of the queryconcept codes, and in an embodiment, the query concept codes aretranslated from a first nomenclature (for example, a proprietarynomenclature) to a second nomenclature or multiple nomenclatures (thenomenclatures of the data to be queried). The query is performed and theresults are received. In an embodiment, the results are translated backinto the first nomenclature and provided to the caregiver, such as viathe user interface. Additional examples of querying operations areprovided in connection to FIG. 4C.

With reference to FIG. 7A, an example finite state machine solver 700,is illustratively depicted, specific to a patient and suitable for useto determine a condition or recommended treatment in accordance withembodiments of the present invention. This example of an instantiatedfinite state machine solver is specific to a patient suffering fromheart failure and other conditions. The finite state machine can beevaluated to determine the patient's specific condition and recommendedtreatment. In this example, content parameters, such as example contenttables 2124, provide parameters specifying the transition conditions foreach state of the finite state machine, while the discretized patientinformation, provides the states of the finite state machine.Accordingly, the states of the finite state machine correspond toconditions of the patient. Based on this, this finite state machinemaybe traversed in order to determine the current state (i.e.,condition, including epochs) of the patient. Based on determining thepatient's state, an appropriate condition program can be selected orgenerated, in some embodiments via an agent. In an embodiment, each ofthe previous states associated with the patient comprises a sensor orpreceptor that may determine the present state of the patient and eachpotential future state of the patient comprises an effector, such asdiscussed in connection to FIG. 1C.

In the example embodiment of FIG. 7A, each vertical column of the finitestate machine corresponds to a different condition-type of the patient,and each state within each column corresponds to a specific condition.For example, one of the columns of the finite state machine correspondsto a DM (diabetes) condition-type. Within this column, three states arepresent: euglycemia, hypoglycemia, and hyperglycemia. Adjacent to eachstate are transition parameters specifying conditions necessary totransition to another state. For example, a patient will be in theHyperglycemia state when the discretized patient information indicatesthat “GLU≥140 mg/dL.” In some instances, evaluating each state of thefinite state machine may require invoking another solver, such as amixed integer solver—a type of linear solver. The output of theevaluation of the finite state machine of FIG. 7A is a determinedcondition or recommended treatment for the patient, and may be used fordetermining or creating a condition program for the patient.

FIGS. 7B-7D illustratively depict past, present, and future potentialpatient states associated with clinical decisions for a patient. Thesedepictions are intended to provide examples of patient event sequences,including past, current, and potential future decision support events,for determining next (or future decision support events associated withthe patient. Additional details are provided in connection to FIGS.4G-4I.

With reference to FIG. 7B, a set or vector of multivariable predicateswhose truth-states are ascertained from information that is contained inthe electronic health record is assembled at time t−1 and comprises asensor pattern that denotes a compound or composite predicate thatmerits attention and decision-making. In an embodiment, the concurrentmaterialization of the predicate pattern vector causes a health careagent (such as a CareDecisions agent) or triggers a software service orprocess to nominate at least one specific CareDecision epoch A at time tand in an embodiment, enqueue it for a qualified user's review such asfor likely decisions and action. Epoch A thus represents a criticaljuncture, in an embodiment.

In one embodiment, the epoch A is not a transient or momentary state;rather, it persists for a finite period that is commensurate withtimeframes that are customary for ordinary, decision-making in healthcare services. Epochs are delimited in time by the onset of sensorpredicate vector elements' values such that a distance for matching thevector against one or more of the set of reference epoch vectors issuitably small, and by the offset of vector elements' values such thatthe distance is larger than a threshold denoting a close or satisfactorymatch. In some embodiments, the offset may be due to the beneficialeffect of a treatment or intervention that has been initiated. In someembodiments, the offset may be due to changes that arise spontaneouslyfor the patient irrespective of any treatment or intervention, or theoffset may be due to the emergence of a preponderance of multivocal andcountervailing evidence from other sources as measured by cronbach alphaor other means such that one or more of the predicates that ledinitially to the epoch's being nominated is overruled. Epochs may be asshort as several seconds or minutes in length in the case of criticalcare and perioperative care or may be as long as several years in thecase of slowly-evolving chronic diseases. In an embodiment, caredecision epochs are defined by parameters 2120 (discussed in connectionto FIG. 1C), and may indicate the specific sensor information (such asclinical conditions (including sequences or patterns of clinicalconditions) and clinical variables (including patient demographicvariables, treatment history and caregiver/health care entity/insuranceinformation) used to determine the decision epoch. It is contemplatedthat in some embodiments, decision epochs are discovered or “learned” bythe mining and processing of patient information such as described inconnection with FIG. 3A. For example, it may be determined that amonggroups of patients with common clinical information, certain sensors orpatterns (including sequences) of sensors frequently lead to certaindecisions points or decision patterns in patient treatment. Accordinglyfor a given patient having a set of sensors in common with otherpatients each of whom experienced a particular decision epoch, aprobability can be determined that the particular decision epoch willapply to the given patient. Based on that probability, one or morerecommendations may be determined for the patient, such as altering thecourse of care for the patient, consulting a clinical specialists, orother modification to treatment.

In an embodiment, the definition of a decision epoch contains the sensorinformation that led to the epoch's evocation. In an embodiment, it alsocontains a CareDecision epoch reference database predicate vector thatthe patient's information matched, and in an embodiment contains aTanimoto or Jaccard or other distance metric denoting the degree ofsimilarity or quality of matching of the patient's sensor vector to thereference vector associated with the nominated epoch. In an embodiment,it further contains the evidence-based content that is clinicallyindicated and recommended in the circumstance that obtains at time t,plus normative content that is data-mined and known to be reasonablyprevalent and associated with desirable outcomes in that circumstance.In an embodiment, it simultaneously contains evidence-based content thatis clinically contraindicated and either absolutely or relativelyproscribed in the circumstance that obtains at time t, plus normativecontent that is data-mined and known to be reasonably prevalent andassociated with undesirable outcomes in that circumstance, particularlywith regard to interactions among therapeutics that may be utilized inother comorbid health conditions. In an embodiment, the normative andprescriptive content, in turn, may provoke obtaining additionalinformation from the patient at t+1, which may include withoutlimitation answers to a patient assessment, such as described inconnection to FIG. 5E, a standardized questionnaire that is pertinent tothe circumstance, observation of circumstance-relevant patient findingsby physical examination, querying the patient (or family member orcaregiver) for additional information, performing one or more diagnostictests, initiating one or more therapeutic maneuvers or procedures orprescribed medications, or temporizing for a circumstance-relevantperiod of time, for example. In some embodiments, these items ofinformation may be referenced in a further sensor pattern, which mayoptionally include one or more sensors that did not arise as aconsequence of the prior decision epoch. In an embodiment, theinformation at time t+1 itself constitutes an induced orpartially-induced set or vector of multivariable predicates that denotesyet another distinct circumstance that evokes a corresponding decisionepoch.

Turning now to FIG. 7C and with reference to the example shown in FIG.7A and the discussion of FIG. 7B, we have in FIG. 7C, an examplecircumstance where at time t−1 the congestive heart failure CHF sensorhas caused ordering and measurement of left ventricular function andejection fraction by ultrasound or other means and has determined thatleft ventricular dysfunction is presently moderate with an ejectionfraction <40% and increased symptoms, including orthopnea and shortnessof breath. In this example, serum electrolytes measured serially up totime t−1 show sodium trending downward over time. Beta-blocker anddiuretics and other medications routinely prescribed in heart failureare ascertained to be at appropriate or near-maximal dosages in thecontext of progression to NYHA Class III severity. The vector of thepatient's sensor predicate values prevailing at time t is matchedagainst a set of decision epochs, which in one embodiment includepre-defined decision epochs (including learned decision epochs). In oneembodiment, a vector of the sensor predicate values is determined byTanimoto or Jaccard or similar metric to be a close match, resulting inthe nomination of the ‘Exacerbation of CHF with LVD and EF <40%’ epoch.In this example, this epoch is, in turn, associated with evokingevidence-based and data-mined normative prescriptive content at timet+1—including running ‘Beck Depression Inventory’ (BDI) and ‘MinnesotaLiving with Heart Failure’ (MLWHF) assessments (or re-running them, inthe case that they have been completed on a previous date more remotethan 14 days before the current date); ordering a mineralocorticoidreceptor antagonist medication, such as spironolactone; and ordering arecurring, periodic measurement of serum potassium and serum creatinine,to monitor and guard against the risks of hyperkalemia and renal injury,respectively, which may be associated with mineralocorticoidmedications. The information collected by running the MLWHF at time t+1may determine that the patient's Activities of Daily Living (ADLs) aresignificantly impaired, leading to the nomination of the ‘Decompensationof ADL’ CareDecision epoch at time t+2. This, in turn, may, with othersensors, cause the evocation of evidence-based or data-mined normativeprescriptive content at time t+3, such as performing the ‘PhysicalSelf-Maintenance Scale’ (PSMS) or other comparable assessment, tofurther characterize and quantify the extent of impairment of ADLs. Itmay also cause the initiation of physiotherapy, occupational therapy,social work or case manager engagement, revision of dietary orders, orfurther review and adjustment of pharmaceutical regimen.

As contrasted with static, standardized clinical pathways or protocols,the timing of the materialization of the epochs and the data-minednormative content have a saliency that reflects frequent relevantitemsets that are prevalent in the mined historical records of similarpatients including recent patients. Rather than exalting the simplisticinsights that are represented in a pathway, the effect is to revealthose decisions and alternative actions that are relevant to “patientslike this, at moments like the present one”, which has the effect ofpromoting optimal timing and intensity and duration of intervention,such as in this example of heart failure management programs withnear-optimal targeting.

FIG. 7D shows a diagram similar to FIGS. 7B and 7C. In the exampleillustratively provided in FIG. 7D, at time t a patient is in aparticular state or condition (here, also a decision epoch) S_(t),having arrived at S_(t) from a previous state S_(t-1). FIG. 7D showsthat at time t+1, at least four possible states are available, with thelikelihood probability that the patient will transition to a particularstate at time t+1, given as P₄, P₁, P₂, and P₃. In an embodiment, thislikelihood or probability is determined based on the comparison ofsimilar patients, such as described in FIG. 7B. FIG. 7D also shows thatif the patient transitions to state 1 (which has a probability of P₁),then at time t+2, the likelihood that the patient will transition toeach of the states shown at time t+2, is given as P₁₋₂, and P₁₋₁. Thusthe probability may also be determined not only for the next state (orcondition, including the evocation of decision epochs) at time t+1, butalso for future states at time t+2, t+3, etc. for the patient.

FIG. 8A-8D depicts illustrative screen displays of an example userinterface for presenting and receiving clinical patient information inaccordance with embodiments of the present invention. FIG. 8B shows anexample of providing a user with additional information by selecting anitem in the TOC (here “growth chart”) for the patient, in comparison toother patients. FIG. 8C presents an example of a decision supportapplication built using the EndPage framework within Cerner PowerChart®.

Additional example embodiments of the invention include: a system,method, and computer readable media having computer-executableinstructions embodied thereon that when executed, facilitate a methodfor providing clinical decision support, the method comprising:receiving patient health data from a healthcare entity, the patienthealth data including a set of clinical concepts encoded in a firstnomenclature; determining a second nomenclature corresponding to thefirst nomenclature; accessing a mapping database of mappings between thefirst nomenclature and the second nomenclature; from the first set ofclinical concepts, determining a corresponding second set of clinicalconcepts encoded in the second nomenclature; accessing a first clinicalcondition program which uses a subset of the clinical concepts in thesecond set of clinical concepts to determine a probability that thepatient has a first clinical condition; based on the first clinicalcondition program, determining that a patient has a probability for thefirst clinical condition; from the mapping database, determining a setof first-condition information including clinical concepts encoded inthe first nomenclature and representing the first clinical condition,the determined probability that the patient has the first clinicalcondition, and the subset of clinical concepts used by the firstcondition program; and providing the first-condition information to thehealth care entity.

Some embodiments of the system, method, and computer readable mediadescribed above further comprise presenting in a user interface, a menuincluding the first clinical condition; and responsive to selecting thefirst clinical condition, presenting: (a) a condition risk scorerepresenting the determined probability that the patient has for thefirst clinical condition; (b) a set of risk factors corresponding to thesubset of clinical concepts used by the first condition program; and (c)for at least a portion of the risk factors, a clinical value for thefactor, the value determined based on information derived from thepatient. In an embodiment of the systems, methods, and computer-readablemedia, described herein, the first clinical condition program isimplemented by one or more software agents. In an embodiment, accessinga mapping database is facilitated by one or more software agents. In anembodiment, at least one of the presented menu, the presented conditionrisk score, or the set of risk factors are dynamically responsive tochanges in the first clinical condition program. In an embodiment,clinical concepts comprise lab results, test results, patientconditions, patient health history, patient demographic information, orother discretized health-related information. In an embodiment, theclinical condition includes one or more of a disease, diagnoses, medicalissue, or medical event; the probability for the first clinicalcondition is a calculated probability that the patient has or willdevelop the first clinical condition based on at least a portion of theset of clinical concepts for the patient; risk factors comprise one ormore clinical variables of health data from population of patients thatare determined to contribute to the likelihood that a given patient hasor will develop the first clinical condition; the clinical variablescorrespond to clinical concepts; a clinical value comprises apatient-specific value corresponding to a clinical variable; the patienthealth information comprises specific lab results, tests, health-relatedfindings, patient conditions, patient history, or otherclinical-information component of a patient's health data; and/or thepresented condition risk score and set of risk factors are dynamicallyresponsive to changes in the first clinical condition program.

Some embodiments of the system, method, and computer readable mediadescribed herein further comprise presenting a condition assessment areafor presenting a contextually-driven assessment based on patientinformation relevant to diagnosing or treating the condition, and forreceiving patient information in response to presenting the assessment,wherein the received information includes one or more clinical conceptsencoded in the first clinical nomenclature. Some embodiments of thesystem, method, and computer readable media described herein furthercomprise presenting a condition assessment area for presenting acontextually-driven assessment based on patient information relevant todiagnosing or treating the condition and determining that patientinformation is absent or stale in the patient health data.

In one aspect of the embodiments described herein, a system, method, orcomputer-readable media having computer-executable instructions embodiedthereon that when executed, is provided for facilitate a method forproviding clinical decision support comprising: receiving a first set ofinformation about a patient, the first set of information including oneor more clinical concepts encoded in a first nomenclature; determining asecond nomenclature corresponding to the first nomenclature; accessing amapping database of mappings between the first nomenclature and thesecond nomenclature; translating at least a portion of the one or moreclinical concepts encoded in the first nomenclature into the secondnomenclature thereby creating a second set of information about thepatient including one or more clinical concepts encoded in the secondnomenclature; accessing a library of clinical condition programs, eachclinical condition program associated with a set of clinical conceptrisk factors encoded in the second nomenclature; determining one or moreclinical conditions associated with the patient based on the second setof patient information and based on at least one of the clinicalcondition programs; translating into the first nomenclature thedetermined one or more clinical conditions and set of risk factorsassociated with the condition program corresponding to each of the oneor more clinical conditions; presenting a user interface including amenu of condition items, each condition item corresponding to thedetermined one or more clinical conditions, wherein the conditions arepresented using the first nomenclature; and responsive to selecting anitem in the menu, displaying a risk score for the patient having thecondition corresponding to the item, and displaying a set of riskfactors relevant to the risk score, wherein the risk factors arepresented in the first nomenclature. In an embodiment of the systems,methods, and computer-readable media, described herein, mapping entriesof the mapping database are determined by one or more software agents.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media having computer-executableinstructions embodied thereon that when executed, facilitate a methodfor providing clinical decision support, the method comprising:receiving patient health data from a healthcare entity, the patienthealth data including a set of clinical concepts encoded in a firstnomenclature; determining a second nomenclature corresponding to thefirst nomenclature; accessing a mapping database of mappings between thefirst nomenclature and the second nomenclature; from the first set ofclinical concepts, determining a corresponding second set of clinicalconcepts encoded in the second nomenclature; accessing a first clinicalcondition program which uses a subset of the clinical concepts in thesecond set of clinical concepts to determine a probability that thepatient has a first clinical condition; based on the first clinicalcondition program, determining that a patient has a probability for thefirst clinical condition; from the mapping database, determining a setof first-condition information including clinical concepts encoded inthe first nomenclature and representing the first clinical condition,the determined probability that the patient has the first clinicalcondition, and the subset of clinical concepts used by the firstcondition program; and providing the first-condition information to thehealth care entity.

Some embodiments described herein further comprise presenting in a userinterface, a menu including the first clinical condition; and responsiveto selecting the first clinical condition, presenting: (a) a conditionrisk score representing the determined probability that the patient hasfor the first clinical condition; (b) a set of risk factorscorresponding to the subset of clinical concepts used by the firstcondition program; and (c) for at least a portion of the risk factors, aclinical value for the factor, the value determined based on informationderived from the patient.

In an embodiment of the systems, methods, and computer-readable media,described herein, the first clinical condition program is implemented byone or more software agents; accessing a mapping database is facilitatedby one or more software agents; at least one of the presented menu, thepresented condition risk score, or the set of risk factors aredynamically responsive to changes in the first clinical conditionprogram; clinical concepts comprise lab results, test results, patientconditions, patient health history, patient demographic information, orother discretized health-related information; the clinical conditionincludes one or more of a disease, diagnoses, medical issue, or medicalevent; the probability for the first clinical condition is a calculatedprobability that the patient has or will develop the first clinicalcondition based on at least a portion of the set of clinical conceptsfor the patient; risk factors comprise one or more clinical variables ofhealth data from population of patients that are determined tocontribute to the likelihood that a given patient has or will developthe first clinical condition; the clinical variables correspond toclinical concepts; a clinical value comprises a patient-specific valuecorresponding to a clinical variable; the patient health informationcomprises specific lab results, tests, health-related findings, patientconditions, patient history, or other clinical-information component ofa patient's health data; and/or the presented condition risk score andset of risk factors are dynamically responsive to changes in the firstclinical condition program.

Some embodiments described herein further comprise presenting acondition assessment area for presenting a contextually-drivenassessment based on patient information relevant to diagnosing ortreating the condition, and for receiving patient information inresponse to presenting the assessment, wherein the received informationincludes one or more clinical concepts encoded in the first clinicalnomenclature. Some embodiments described herein further comprisepresenting a condition assessment area for presenting acontextually-driven assessment based on patient information relevant todiagnosing or treating the condition and determining that patientinformation is absent or stale in the patient health data.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: receiving a first set of information abouta patient, the first set of information including one or more clinicalconcepts encoded in a first nomenclature; determining a secondnomenclature corresponding to the first nomenclature; accessing amapping database of mappings between the first nomenclature and thesecond nomenclature; translating at least a portion of the one or moreclinical concepts encoded in the first nomenclature into the secondnomenclature thereby creating a second set of information about thepatient including one or more clinical concepts encoded in the secondnomenclature; accessing a library of clinical condition programs, eachclinical condition program associated with a set of clinical conceptrisk factors encoded in the second nomenclature; determining one or moreclinical conditions associated with the patient based on the second setof patient information and based on at least one of the clinicalcondition programs; translating into the first nomenclature thedetermined one or more clinical conditions and set of risk factorsassociated with the condition program corresponding to each of the oneor more clinical conditions; presenting a user interface including amenu of condition items, each condition item corresponding to thedetermined one or more clinical conditions, wherein the conditions arepresented using the first nomenclature; and responsive to selecting anitem in the menu, displaying a risk score for the patient having thecondition corresponding to the item, and displaying a set of riskfactors relevant to the risk score, wherein the risk factors arepresented in the first nomenclature. In an embodiment, mapping entriesof the mapping database are determined by one or more software agents.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: receiving a search query comprising one ormore search terms; from a first nomenclature of clinical concept-codes,determining a first nomenclature concept-code associated with each ofthe one or more search terms; from a second nomenclature of clinicalconcept-codes, determining a second nomenclature concept-codecorresponding to a first nomenclature concept code, for each of thedetermined first-nomenclature concept codes, thereby forming a set ofsecond nomenclature concept-codes; performing a search query based onthe set of second nomenclature concept-codes; receiving from the searchquery a set of search results including result concepts that are codedbased on the second nomenclature of clinical concept-codes; for each ofthe result concepts, determining a concept-code from the firstnomenclature that corresponds to result concept code based on the secondnomenclature; and providing the set of search results including theresult concepts, wherein the result concepts are coded based on thefirst nomenclature of concept codes.

In an embodiment of the systems, methods, and computer-readable media,described herein, determining a second nomenclature concept-codecorresponding to a first nomenclature concept code comprises: accessinga mapping database of mappings between the first nomenclature and thesecond nomenclature; and translating the first nomenclature concept codeto the second nomenclature concept code based information in the mappingdatabase. In an embodiment of the systems, methods, andcomputer-readable media, described herein, the search query comprisesinformation about a first patient and the set of search results includeinformation associated with one or more second patients having at leasta portion of clinical information in common with the first patient,and/or the search query comprises information about a first care giveror first health care entity associated with a first patient and the setof search results include information associated with one or more secondcare givers or second health care entities having at least a portion ofinformation in common with the first care giver or first health careentity. In an embodiment, the search query is received by a userinterface. In an embodiment, the user interface comprises a set ofuser-interface elements including: (a) a clinical conditions menu forpresenting a set of clinical conditions, each of the presented clinicalconditions having a corresponding clinical condition program used fordetermining a condition risk score indicative of a probability that thepatient has the clinical condition, wherein the presented set ofclinical conditions are determined based on a treatment-session context;(b) a clinical condition risk area for presenting, responsive to aselection of a given clinical condition from the menu: (i) a conditionrisk score representing the patient's risk for having the given clinicalcondition, the score determined from the corresponding clinicalcondition program; (ii) a set of risk factors used by the clinicalcondition program for determining the condition risk score; and (iii)for at least a portion of the risk factors, a clinical value for thefactor, the value determined from information derived from the patient;and (c) a clinical information area for presenting a plurality ofclinical information elements associated with the given clinicalcondition, the elements populated with clinical values for the patientfrom a health record, wherein the plurality of elements presented isdetermined based on the condition care program and organizationallypresented based on a treatment-session context.

In an embodiment of the systems, methods, and computer-readable media,described herein, the search query is received by selecting one or moreuser interface elements or portions of one or more user interfaceelements, and wherein the one or more user interface elements orportions of one or more user interface elements comprise the one or moresearch terms, and in an embodiment, selecting one or more user interfaceelements comprises one or more of clicking, right-clicking, lassoing,touching, or hovering over the one or more user interface elements orportions of one or more user interface elements.

In an embodiment of the systems, methods, and computer-readable media,described herein, the search query comprises information including oneor more clinical conditions associated with first patient and the set ofsearch results include information including one or more clinicalconditions associated with a second patient having at least a portion ofclinical information in common with the first patient; a medical eventcomprises one or more of admission or readmission to a health carefacility, change in patient health status, sudden injury, patienttreatment, patient reaction, patient response, treatment interaction, orother health care related event affecting the first patient; and/or thesearch query comprises information including one or more possible futureclinical conditions associated with first patient and the set of searchresults include information including one or more clinical conditionsassociated with a second patient having at least a portion of clinicalinformation in common with the first patient.

In one aspect of the embodiments described herein, there is provided asystem for providing clinical decision support, comprising: one or moreprocessors coupled to a computer storage medium, the computer storagemedium having stored thereon a plurality of computer software componentsexecutable by the processor, the computer software componentscomprising: a query component that receives a search query comprisingone or more search terms; a code component that determines, from a firstnomenclature of clinical concept-codes, a first nomenclatureconcept-code associated with each of the one or more search terms;wherein the code component further determines, from a secondnomenclature of clinical concept-codes, a second nomenclatureconcept-code corresponding to a first nomenclature concept-code, foreach of the determined first-nomenclature concept codes, thereby forminga set of second nomenclature concept-codes; a search component thatperforms a second search query based on the set of second nomenclatureconcept-codes; a results component that receives, from the searchcomponent, a set of search results including result concepts that arecoded based on the second nomenclature of clinical concept-codes; aconcept-code component that determines, for each of the result concepts,a concept-code from the first nomenclature that corresponds to resultconcept code based on the second nomenclature; and a presentationcomponent that provides the set of search results including the resultconcepts, wherein the result concepts are coded based on the firstnomenclature of concept codes.

Some embodiments described herein further comprise a menu component thatpresents a set of clinical conditions, each of the presented clinicalconditions having a corresponding clinical condition program used fordetermining a condition risk score indicative of a probability that thepatient has the clinical condition, wherein the presented set ofclinical conditions are determined based on a treatment-session context.

Some embodiments described herein further comprise a risk component thatpresents, responsive to a selection of a given clinical conditionpresented by the menu component: a condition risk score representing thepatient's risk for having the given clinical condition, the scoredetermined from the corresponding clinical condition program; a set ofrisk factors used by the clinical condition program for determining thecondition risk score; and for at least a portion of the risk factors, aclinical value for the factor, the value determined from informationderived from the patient; and a clinical information area for presentinga plurality of clinical information elements associated with the givenclinical condition, the elements populated with clinical values for thepatient from a health record, wherein the plurality of elementspresented is determined based on the condition care program andorganizationally presented based on a treatment-session context.

Some embodiments described herein further comprise a mapping componentthat accesses a mapping database of mappings between the firstnomenclature and the second nomenclature; and/or a translation componentthat translates the first nomenclature concept code to the secondnomenclature concept code based information in the mapping database.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: presenting a first clinical user interfaceassociated with a first patient having a condition, the first clinicaluser interface including: (a) a clinical information area having aplurality of clinical information elements, at least a portion of theplurality of clinical information elements being populated with storedclinical information associated with the first patient; (b) a clinicalrecommendations area for presenting a clinical recommendation associatedwith the condition; and/or (c) a user-input area for receiving usercommands. In an embodiment, further comprising: receiving a command toinitiate a clinical decision support event associated with the firstpatient; associating the clinical decision support event with thecondition; determining a change in the condition of first patient;identifying a second patient having the condition of the first patient;and based on the clinical decision support event associated with thefirst patient and the determined change in the condition of the firstpatient, determining a clinical recommendation for the second patient.

Some embodiments described herein further comprise presenting a secondclinical user interface associated with the second patient; andpresenting the clinical recommendation for the second patient in thesecond user interface. Some embodiments described herein furthercomprise determining a user's response to the recommendation for thesecond patient and logging information indicating the user's response.

In an embodiment of the systems, methods, and computer-readable media,described herein, the clinical recommendation includes clinical conceptsencoded using a proprietary vocabulary; the received command is receivedin a first nomenclature and the clinical recommendation includesclinical concepts encoded in a second nomenclature; and/or clinicalconcepts associated with the first patient are encoded in a firstnomenclature and clinical concepts associated with the second patientare encoded in a second nomenclature. In an embodiment of the systems,methods, and computer-readable media, described herein, identifying asecond patient having a condition of the first patient comprises:determining a first set of clinical concept codes associated with thecondition of the first patient; the first set of codes being encoded ina first nomenclature; determining a second set of clinical concept codesthat correspond to the first set of concept codes, the second set ofcodes being encoded in a second nomenclature; and from a set of patienthealth records for one or more patients other than the first patient,identifying a second patient having clinical concept codes matching thesecond set of clinical concept codes.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: presenting a first clinical user interfaceassociated with a first patient having a condition, the first clinicaluser interface including: (a) a clinical information area having aplurality of clinical information elements, at least a portion of theplurality of clinical information elements being populated with storedclinical information associated with the first patient; (b) a clinicalrecommendations area for presenting a clinical recommendation associatedwith the condition; and/or (c) a user-input area for receiving usercommands. In an embodiment, further comprises: receiving a command toinitiate a clinical support event associated with a first patient, thecommand being encoded using a first clinical nomenclature; associatingthe clinical decision event with a clinical condition; determining achange in the first patient's condition; storing a first set ofinformation for the first patient indicative of the event, theassociation with the condition, and the change in condition, theinformation encoded in a first clinical nomenclature; determining asecond nomenclature corresponding to the first nomenclature; translatingthe first set of information encoded in the first nomenclature into asecond set of information encoded in the second nomenclature; andstoring the second set of information in a health records database.

Some embodiments described herein further comprise removing identifyinginformation from the second set of information thereby formingde-identified information. In an embodiment of the systems, methods, andcomputer-readable media described herein, translating the first set ofinformation encoded in the first nomenclature into a second set ofinformation encoded in the second nomenclature comprises accessing amapping database of mappings between the first nomenclature and thesecond nomenclature. An embodiment further comprises: identifying asecond patient having the condition of the first patient; based on thesecond set of information, determining a clinical recommendation for thesecond patient; and presenting the clinical recommendation for thesecond patient.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: identifying a first set of patients from aset of patient caregivers, the first set of patients having in common afirst set of clinical concepts; monitoring the set of patients todetermine a change in condition of each patient in the first set ofpatients; identifying a second set of patients from a target caregiver;the second set of patients having in common a the first set of clinicalconcepts; for each patient in the second set of patients: (a) monitoringthe patient to determine a change in condition; and (b) comparing achange in condition of the patient with changes in condition of patientsin the first set of patients; based on the comparison for each patientin the second set of patients, determining a treatment score for thetarget caregiver. In an embodiment, the comparison is based on anaverage change in patient conditions for patients in the first set ofpatients; monitoring comprises tracking treatment costs (includingfinancial, risk, and resources) associated with the patient, and thecomparison includes a comparison of treatment cost information.

In one aspect of the embodiments described herein, a system is providedfor clinical decision support comprising: one or more processors coupledto a computer storage medium, the computer storage medium having storedthereon a plurality of computer software components executable by theprocessor, the computer software components comprising: a first setcomponent that identifies a first set of patients from a set of patientcaregivers, the first set of patients having in common a first set ofclinical concepts; a first monitor component that monitors the first setof patients to determine a change in condition of each patient in thefirst set of patients; a second set component that identifies a secondset of patients from a target caregiver, the second set of patientshaving in common the first set of clinical concepts; a second monitorcomponent that, for each patient in the second set of patients, monitorsthe patient to determine a change in condition; a comparison componentthat compares a change in condition of each patient in the second set ofpatients with changes in condition of patients in the first set ofpatients; and a treatment score component that, based on the comparisonfor each patient in the second set of patients, determines a treatmentscore for the target caregiver.

Some embodiments described herein further comprise an event componentthat receives a command to initiate a clinical decision support eventassociated with the first set of patients; an association component thatassociates the clinical decision support event with the first set ofclinical concepts; and a recommendation component that, based on theclinical decision support event associated with the first set ofpatients and the changes in condition of patients in the first set ofpatients, determines a clinical recommendation for each patient in thesecond set of patients. Some embodiments further comprise a firstnomenclature component that stores a first set of information for thefirst set of patients indicative of the clinical decision support event,the association with the first set of clinical concepts, and the changesin condition of patients in the first set of patients, the first set ofinformation encoded in a first nomenclature; and some embodimentsfurther comprise a second nomenclature component that determines asecond nomenclature corresponding to the first nomenclature. Someembodiments further comprise: a translation component that translatesthe first set of information encoded in the first nomenclature into asecond set of information encoded in the second nomenclature, whereinthe translation component accesses a mapping database of mappingsbetween the first nomenclature and the second nomenclature. Someembodiments further comprise a store component that stores the secondset of information in a health records database, wherein identifyinginformation is removed from the second set of information therebyforming de-identified information.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: receiving unstructured health-related dataassociated with a first patient, the health related data including adiscrete element; identifying a preliminary match between the discreteelement and a candidate clinical concept encoded in a firstnomenclature; determining a first-patient set of clinical conceptsassociated with the first patient; receiving structured health-relateddata associated with a population of patients the structured healthrelated data including, for each patient in the population, a set ofclinical concepts matching the first-patient set of clinical concepts;determining a likelihood of patients in the population also having thecandidate clinical concept; and based on the determined likelihood,determining that the discrete element matches the candidate clinicalconcept. Some embodiments described herein further comprise determiningthe discrete element from the received unstructured health related datausing a natural language processing service.

In an embodiment of the systems, methods, and computer-readable media,described herein, natural language processing service uses anopen-source natural language processing system; the natural languageprocessing service is carried out by one or more software agents; thereceived unstructured health-related data comprises received audio orspeech data; the received unstructured health-related data comprisesdata from a monitor or sensor associated with the first patient; and/orthe received unstructured health-related data is received from the firstpatient. Some embodiments described herein further comprise presentinginformation about the discrete element and candidate clinical concept toa user for confirming a match; and receiving a confirmation from theuser that the discrete element matches the candidate clinical concept.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: receiving unstructured patient healthrelated data; processing the unstructured data to determine one or moreclinical concepts from the unstructured data, at least one of theconcepts corresponding to a clinical condition of the patient, whereinthe concepts encoded are in a first nomenclature; associating theclinical concepts with each other, thereby forming a set of associatedclinical concepts; determining a second nomenclature corresponding tothe first nomenclature; translating the set of associated clinicalconcepts into the second nomenclature; storing the set of associatedclinical concepts in the second nomenclature; from a database of healthrecords, determining one or more similar sets of associated clinicalconcepts; and designating the clinical concepts as a contextual ontologyfor the clinical condition.

In an embodiment of the systems, methods, and computer-readable media,described herein, processing the unstructured data to determine one ormore clinical concepts comprises using a natural language processingservice; the natural language processing service uses an open-sourcenatural language processing system; and/or the natural languageprocessing service is carried out by one or more software agents.

In one aspect of the embodiments described herein, there is provided asystem for providing clinical decision support comprising one or moreprocessors coupled to a computer storage medium, the computer storagemedium having stored thereon a plurality of computer software componentsexecutable by the processor, the computer software componentscomprising: an unstructured data component that receives unstructuredhealth-related data associated with a first patient, the health relateddata including a discrete element; a match component that identifies apreliminary match between the discrete element and a candidate clinicalconcept encoded in a first nomenclature; a concept component thatdetermines a first-patient set of clinical concepts associated with thefirst patient; a structured data component that receives structuredhealth-related data associated with a population of patients thestructured health related data including, for each patient in thepopulation, a set of clinical concepts matching the first-patient set ofclinical concepts; a likelihood component that determines a likelihoodof patients in the population also having the candidate clinicalconcept; and a determination component that, based on the determinedlikelihood, determines that the discrete element matches the candidateclinical concept.

Some embodiments described herein further comprise a processingcomponent that determines the discrete element from the receivedunstructured health related data, and in an embodiment, the processingcomponent utilizes natural language processing. Some embodimentsdescribed herein further comprise a presentation component that presentsinformation about the discrete element and candidate clinical concept toa user for confirming a match; and/or a confirmation component thatreceives a confirmation from the user that the discrete element matchesthe candidate clinical concept.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: from a first sequence of past clinicalevents relating to a patient; determining a clinical event patternassociated with the patient; from health record data associated with apopulation of patients, determining a set of patient health records thatinclude the clinical event pattern; and for each member in the set ofpatient health records, determining a second sequence of clinical healthevents succeeding the clinical event pattern thereby forming a set ofsecond sequences of clinical health events.

Some embodiments described herein further comprise based on the set ofsecond sequences, determining a likelihood of a future health care eventfor the patient; recommending a health care action to take based on thelikelihood of the future event; and/or for each member of the set ofsecond sequences of clinical health events, determining a costassociated with the sequence; determining a measure of improvement inthe health condition at the end of the sequence as indicated by thehealth record associated with each sequence; and ranking the members ofthe set of sequences based on the measure of improvement and cost. Someembodiments described herein further comprise recommending a caretreatment sequence for the patient based on the ranking.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: determine that patient has a probabilityfor clinical event; determining that patient is missing clinical dataneeded for increasing or decreasing the determined event probability;determining that clinical information is not currently available for oneor more of the clinical information elements relevant to the decisionsupport event; identifying a set of patients having similar clinicalconcepts associated with the condition; imputing values for targetpatient's clinical data based on set of patients clinical data; andusing the imputed data to determine an updated probability for theclinical event. Some embodiments described herein further compriseproviding an indication of the difference in target patient's EHRbetween actual patient data and imputed data, such as presenting imputeddata in a color different than the actual patient data.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: for a patient, determining a pattern ofclinical events (which may include one or more epochs); from apopulation of patients, identifying a set of patient records withsimilar pattern of clinical events; for each record, determining aseries of events preceding the pattern thereby forming a future eventseries for each record; and designating frequently occurring event as atrigger. In an embodiment, two or more records of the set of records arein different nomenclatures.

Some embodiments described herein further comprise clustering thepatient records by their change in condition. Some embodiments describedherein further comprise for each cluster of records, determining afrequently occurring events among the future event series, therebyforming a frequently occurring set associated with each record cluster;and/or designating frequently occurring event a critical juncture, orcritical epoch. Some embodiments described herein further comprisealerting caregiver of critical juncture thereby facilitating increasedattention may be administered to patient. Some embodiments describedherein further comprise determining additional clinical information forthe patient to determine a probability that a future event will occur.In an embodiment, determining additional clinical information includesproviding a patient assessment such as described in connection to FIGS.5E, for determining at least a portion of the additional clinicalinformation, wherein the patient assessment is generated based on atreatment-session context.

Some embodiments described herein further comprise based on thefrequently occurring events for each cluster, determining arecommendation for a target patient; selecting recommended care-pathwayto execute, wherein the care-pathway comprises a treatment program;and/or determining a degree of similarity between a target patient andthe set of patient records; and based on the degree of similarity,determining a probability that the target patient will have one or moreof the events in the future event series. Some embodiments describedherein further comprise determining a cost associated with each futureevent series.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: receiving a first set of clinicalinformation associated with a patient from a data store; based on thefirst set of clinical information, determining a preliminary likelihoodof a clinical decision support event being associated with the patient;determining that additional clinical information is not currentlyavailable for one or more clinical information factors relevant to theclinical decision support event; providing or generating a patientassessment for determining at least a portion of the additional clinicalinformation, wherein the patient assessment is generated based on atreatment-session context; and providing the patient assessment to ahealth care provider treating the patient.

In an embodiment of the systems, methods, and computer-readable media,described herein, treatment-session context is determined based on atleast one of the health care provider's role or clinical specialty, aclinical treatment-session venue, and the clinical decision supportevent, such as the clinical condition(s) of the patient. Someembodiments described herein further comprise receiving the at least aportion of the additional clinical information in response to providingthe assessment, and updating the likelihood of a clinical decisionsupport event based on the received additional clinical information;determining clinical advice or a clinical recommendation (such asdescribed in connection to FIG. 1E), based on the likelihood of aclinical decision support event; and/or receiving the at least a portionof the additional clinical information in response to providing theassessment, and updating the determined clinical advice based on thereceived additional clinical information.

In an embodiment of the systems, methods, and computer-readable media,described herein, the assessment is provided via a graphical userinterface; the assessment comprises a series of questions, wherein ananswer for a first question, received via the user interface, determinesa subsequent question in the series of questions; and/or the additionalclinical information includes clinical concepts encoded in a firstnomenclature. Some embodiments described herein further comprisedetermining clinical advice or recommendation based on the receivedanswer to a first question or series of questions, and presenting theclinical advice in proximity to the first question or series ofquestions.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: receiving a first set of clinicalinformation associated with a patient from a data store; based on thefirst set of clinical information, determining a likelihood of aclinical decision support event being associated with the patient;accessing an assessment associated with the clinical decision supportevent, the assessment including a set of patient related questions;determining from the set of questions a portion of the questions toinclude in a questionnaire, based on a treatment-session context and thefirst set of clinical information; generating a user interface forpresenting the questionnaire; presenting the user interface to a user;and receiving a set of answers, via one or more clinical informationelements of the user interface, in response to the portion of questionsin the questionnaire. Some embodiments described herein further comprisebased on the set of answers, determining an additional portion ofquestions from the set of questions to include in the questionnaire, andpresenting the additional portion of questions.

Some embodiments described herein further comprise receiving a secondset of answers to the additional portion of questions, determiningclinical advice based on the first and second answers and the first setof clinical information; and presenting the clinical advice to the user.Some embodiments described herein further comprise determining aclinical concept code associated with a first question and a firstanswer received in response to the first question; and storing theconcept code in the first set of clinical information.

In an embodiment of the systems, methods, and computer-readable media,described herein, the set of questions are determined based on healthrecords for a population of patients, wherein each health recordincludes information corresponding to the clinical decision supportevent. In an embodiment, the set of questions is further determinedbased on frequently occurring clinical concepts in the health records;and/or the set of questions is determined by a health care softwareagent.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: receiving a target set of clinicalinformation associated with a target population of patients from a firstset of records of a first health-records system, the target set ofclinical information including codified clinical concepts; receiving areference set of clinical information associated with a referencepopulation of patients from a second set of records of a secondhealth-records system, the reference set of clinical informationincluding codified clinical concepts; based on the reference set ofclinical information, determining a clinical decision support eventcommon to the records in the reference population of patients;determining frequent item-sets of the clinical concepts associated withthe reference population; associating the frequent item-sets of clinicalconcepts with the clinical decision support event; performing astatistical comparison between the frequent item-sets and the clinicalconcepts of the target set of clinical information to determining astatistical measure of association between the frequent item-sets andthe clinical concepts of the target set; and based on the statisticalmeasure of association, determining a probability that the targetpopulation of patients includes the clinical decision support event.

In an embodiment of the systems, methods, and computer-readable media,described herein, performing a statistical comparison comprises:performing a distance based, similarity, or cluster-based matching ofthe frequent item-sets and the target set of clinical information todetermine one or more clusters; determining at least one measurequantifying difference for at least one cluster; and determining astatistical measure of association based on the quantifying difference.In an embodiment, the clinical decision support event comprises a healthcondition, combination or health conditions or clinical procedures;clinical concepts are codified using a standardized clinicalnomenclature; and/or the target and reference sets of clinicalinformation are encoded using different nomenclatures.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media for providing clinicaldecision support, comprising: receiving a reference set of clinicalinformation associated with a reference population of patients from aplurality of health-records systems; the reference set of clinicalinformation including codified clinical concepts; based on the referenceset of clinical information, determining a clinical decision supportevent common to patients in the reference population of patients; fromthe reference set of clinical information, determining one or more setsof frequently-occurring clinical concepts; and associating the one ormore sets of frequently occurring clinical concepts with the clinicaldecision support event thereby forming one or more event indicators forthe clinical decision support event.

Some embodiments described herein further comprise receiving a set ofclinical information associated with a first patient, the clinicalinformation including codified clinical concepts; determining a numberof the indicators in set of clinical information; and based on thenumber of indicators, determining a likelihood that the first patienthas the clinical decision support event. Some embodiments describedherein further comprise presenting the determined likelihood to a user;determining a recommended clinical order for the patient based on thedetermined likelihood that the first patient has the clinical decisionsupport event; generating an update for a condition care programassociated with the clinical decision support event, the updateincluding the one or more event indicators; determining the presence ofthe one or more event indicators in a specific patient's health record;and based on the determined presence of the one or more eventindicators, determining a probability that a specific patient has aclinical decision support event.

In an embodiment of the systems, methods, and computer-readable media,described herein, the recommended clinical order is determined based onthe reference set of clinical information associated with a referencepopulation; and/or determining one or more sets of frequently-occurringclinical concepts is determined using a software agent.

In one aspect of the embodiments described herein, there is provided asystem, method, or computer-readable media having computer usableinstructions embodied thereon for presenting one or more user interfacesfor facilitating clinical decision support, the one or more userinterfaces comprising: (a) a clinical conditions menu for presenting aset of clinical conditions, each of the presented clinical conditionshaving a corresponding clinical condition program used for determining acondition risk score indicative of a probability that the patient hasthe clinical condition, wherein the presented set of clinical conditionsare determined based on a treatment-session context; (b) a clinicalcondition risk area for presenting, responsive to a selection of a givenclinical condition from the menu: (i) a condition risk scorerepresenting the patient's risk for having the given clinical condition,the score determined from the corresponding clinical condition program;(ii) a set of risk factors used by the clinical condition program fordetermining the condition risk score; and (iii) for at least a portionof the risk factors, a clinical value for the factor, the valuedetermined from information derived from the patient; and/or (c) aclinical information area for presenting a plurality of clinicalinformation elements associated with the given clinical condition, theelements populated with clinical values for the patient from a healthrecord, wherein the plurality of elements presented is determined basedon the condition care program and organizationally presented based on atreatment-session context.

In an embodiment of the systems, methods, and computer-readable media,described herein, the clinical condition program is accessed from aremote server, and may be downloaded to the client, may reside on theserver, or operate in the cloud. In an embodiment, the condition programis based on healthcare information obtained from a plurality of patienthealth records from at least two record systems having distinct clinicalnomenclatures. In an embodiment the presented condition risk score andset of risk factors are dynamically responsive to changes in thecorresponding condition care program. In an embodiment, the clinicalinformation elements are presented in a proprietary clinicalnomenclature, and in an embodiment, the treatment-session context isbased on at least one of a user-caregiver's clinical specialty, aclinical treatment venue, the given clinical condition, and thecondition care program.

Some embodiments described herein further comprise a conditionassessment area for presenting a contextually-driven assessment based onpatient information relevant to diagnosing or treating a condition andreceiving patient information in response to presenting the assessment,wherein the received information including one or more clinical conceptsencoded in a first clinical nomenclature. Some embodiments describedherein further comprise a condition assessment area for presenting acontextually-driven assessment based on patient information relevant todiagnosing or treating a condition and determining the patientinformation is absent or stale in the patient health data.

Some embodiments of the systems, methods, and computer-readable media,described herein further comprise determining a change in the set ofrisk factors used for determining a condition risk score of a condition;based on the determined change, updating the condition programcorresponding to the condition; and dynamically updating the clinicalcondition risk area of the one or more user interfaces, in response toupdating the condition program. In an embodiment updating the clinicalcondition risk area includes updating the presented condition risk scoreor presented set of risk factors; and/or determining a change in the setof risk factors comprises determining the set of risk factors includes anew or additional risk factor. Some embodiments described herein furthercomprise displaying an indication that the presented condition riskscore or presented set of risk factors have changed; and someembodiments further comprise displaying an indication that a new riskfactor has been added. In an embodiment the presented menu isdynamically responsive to changes in condition care programs or changesin the patient's clinical information associated with the patient.

Some embodiments of the systems, methods, and computer-readable media,described herein further comprise determining that a set of patient datacorresponding to the determined change in the set of risk factors usedfor determining the condition risk score is absent in the health record;and providing a notice to a caregiver that the set of patient data isnot in the health record. Some embodiments further comprise: providingor generating an assessment for obtaining the set of patient data; andpresenting the assessment in a condition assessment area of the userinterface.

Some embodiments of the systems, methods, and computer-readable media,described herein further comprise determining that a set of clinicalvalues for the patient corresponding to the determined change in the setof risk factors used for determining the condition risk score is absentin the health record; and imputing values for the absent set of clinicalvalues for the patient based on a second set of clinical values ofhealth records of a plurality of other patients having a set of clinicalconcepts associated with the condition in common with the patient.

Although the invention has been described with reference to theembodiments illustrated in the attached drawing figures, it is notedthat substitutions may be made and equivalents employed herein withoutdeparting from the scope of the invention as recited in the claims. Forexample, additional steps may be added and steps omitted withoutdeparting from the scope of the invention.

Many different arrangements of the various components depicted, as wellas components not shown, are possible without departing from the spiritand scope of the present invention. Embodiments of the invention havebeen described with the intent to be illustrative rather thanrestrictive. Alternative embodiments will become apparent to thoseskilled in the art that do not depart from its scope. A skilled artisanmay develop alternative means of implementing the aforementionedimprovements without departing from the scope of the invention.

It will be understood that certain features and subcombinations are ofutility and may be employed without reference to other features andsubcombinations and are contemplated within the scope of the claims. Notall steps listed in the various figures need be carried out in thespecific order described.

What is claimed is:
 1. A system comprising one or more processorscoupled to a memory, the one or more processors configured to perform aplurality of operations, the operations comprising: storing a first setof clinical information in a coded clinical format about a condition ofa target patient in one or more network-based non-transitory storagedevices having a collection of medical records stored thereon, whereinthe first set of clinical information includes an initial preliminarylikelihood and one or more risk factors both associated with a clinicaldecision support event; determining, based on a condition care programassociated with the clinical decision support event, that a first set ofrisk factors of the one or more risk factors are absent from the firstset of clinical information; generating, using machine learning executedby the one or more processors, a patient assessment for the clinicaldecision support event, wherein the patient assessment includes at leastone question-answer pair corresponding to one or more coded clinicalconcepts in the coded clinical format, and wherein the machine learningdetermines at least one question of the at least one question-answerpair from a mapped content of patient health records from the one ormore network-based non-transitory storage devices; receiving an updateto the first set of clinical information, the update comprising anon-coded clinical format response to the at least one question-answerpair; converting the non-coded clinical format response into a convertedresponse in the coded clinical format for input into the condition careprogram to update the first set of risk factors; and updating the mappedcontent used by the machine learning to determine the at least onequestion, based on the converted response, wherein the mapped content isupdated using the coded clinical format.
 2. The system of claim 1,wherein the operations further comprise: dynamically updating the firstset of risk factors with a second set of risk factors; automaticallygenerating a message including an updated preliminary likelihood of theclinical decision support event, generated at least in part on the firstset of clinical information and the second set of risk factorsdetermined from the converted response; and transmitting the message toa clinician interface of a clinician device to display the updatedpreliminary likelihood.
 3. The system of claim 2, wherein the operationsfurther comprise: directing the message to a clinician based on anassociation of a role of the clinician to a clinical concept associatedwith at least one answer to the at least one question-answer pair. 4.The system of claim 1, wherein operations further comprise storing theconverted response in the collection of medical records in the codedclinical format.
 5. The system of claim 1, wherein the patient healthrecords include indications of concepts that frequently occur inassociation with a particular condition.
 6. The system of claim 1,wherein the non-coded clinical format response comprises a naturallanguage response.
 7. One or more computer storage devices storinginstructions that, when used by one or more processors, cause the one ormore processors to perform a plurality of operations, the operationscomprising: storing a set of clinical information in a coded clinicalformat about a condition of a target patient in one or morenetwork-based non-transitory storage devices having a collection ofmedical records stored thereon, wherein the set of clinical informationincludes an initial preliminary likelihood and one or more risk factorsboth associated with a clinical decision support event; determining afirst set of risk factors not included in the set of clinicalinformation; generating, using machine learning executed by the one ormore processors, a patient assessment for the clinical decision supportevent, wherein the patient assessment includes at least onequestion-answer pair corresponding to one or more coded clinicalconcepts in the coded clinical format, and wherein the machine learningdetermines at least one question of the at least one question-answerpair from a mapped content of patient health records from the one ormore network-based non-transitory storage devices; receiving an updateto the set of clinical information, the update comprising a non-codedclinical format response to the at least one question-answer pair;converting the non-coded clinical format response into a convertedresponse in the coded clinical format and updating the first set of riskfactors based on the converted response; and updating the mapped contentused by the machine learning to determine the at least one question,based on the converted response, wherein the mapped content is updatedusing the coded clinical format.
 8. The one or more computer storagedevices of claim 7, wherein the operations further comprise: dynamicallyupdating the first set of risk factors with a second set of riskfactors; automatically generating a message including an updatedpreliminary likelihood of the clinical decision support event, generatedat least in part on the set of clinical information and the second setof risk factors determined from the converted response; and transmittingthe message to a clinician interface of a clinician device to displaythe updated preliminary likelihood.
 9. The one or more computer storagedevices of claim 7, wherein the operations further comprise: causing aclient device to display a graphical user interface comprising anassessment area, wherein the patient assessment is presented in theassessment area.
 10. The one or more computer storage devices of claim7, wherein a set of questions for the patient assessment are determinedbased on health records for a population of patients, and wherein eachhealth record includes information corresponding to the clinicaldecision support event.
 11. A method, comprising: storing a first set ofclinical information in a coded clinical format about a condition of atarget patient in one or more network-based non-transitory storagedevices having a collection of medical records stored thereon, whereinthe first set of clinical information includes an initial preliminarylikelihood and one or more risk factors both associated with a clinicaldecision support event; determining, based on a condition care programassociated with the clinical decision support event, that a first set ofrisk factors of the one or more risk factors are absent from the firstset of clinical information; generating, using machine learning executedby one or more processors, a patient assessment for the clinicaldecision support event, wherein the patient assessment includes at leastone question-answer pair corresponding to one or more coded clinicalconcepts in the coded clinical format, and wherein the machine learningdetermines at least one question of the at least one question-answerpair from a mapped content of patient health records from the one ormore network-based non-transitory storage devices; receiving an updateto the first set of clinical information, the update comprising anon-coded clinical format response to the at least one question-answerpair; converting the non-coded clinical format response into a convertedresponse in the coded clinical format for input into the condition careprogram to update the first set of risk factors; and updating the mappedcontent used by the machine learning to determine the at least onequestion, based on the converted response, wherein the mapped content isupdated using the coded clinical format.
 12. The method of claim 11,further comprising dynamically updating the first set of risk factorswith a second set of risk factors.
 13. The method of claim 12, furthercomprising: automatically generating a message including an updatedpreliminary likelihood of the clinical decision support event, generatedat least in part on the first set of clinical information and the secondset of risk factors determined from the converted response.
 14. Themethod of claim 13, further comprising transmitting the message to aclinician interface of a clinician device to display the updatedpreliminary likelihood.
 15. The method of claim 14, further comprising:directing the message to a clinician based on an association of a roleof the clinician to a clinical concept associated with at least oneanswer to the at least one question-answer pair.
 16. The method of claim11, further comprising storing the converted response in the collectionof medical records in the coded clinical format.
 17. The method of claim11, wherein the patient health records include indications of conceptsthat frequently occur in association with a particular condition. 18.The method of claim 11, wherein the non-coded clinical format responsecomprises a natural language response.
 19. The method of claim 11,wherein a set of questions for the patient assessment is determinedbased on health records for a population of patients, and wherein eachhealth record includes information corresponding to the clinicaldecision support event.
 20. The method of claim 11, further comprisingcausing a client device to display a graphical user interface comprisingan assessment area, wherein the patient assessment is presented in theassessment area.